Details on the experiments, including the selected datasets, the evaluation criteria, and the performance of all of the measures, were reported in this section. To obtain a convincing evaluation of EMI, we compared it with the following measures:
For this purpose, the codes were implemented in Python 3.7 and experiments were performed on a computer with an Intel Core i5 2.4 GHz processor and 8 GB RAM.
4.2. Evaluation Criteria
In this experiment, the monotonicity relation [
49], as defined in Equation (30), was employed to evaluate the discriminability of a ranking measure. A ranking algorithm will be better if a few nodes are listed in the same rank.
where
is the number of different ranks in a ranking list
R, and
is the number of nodes that have been listed in the same rank
r. Obviously, if all of the nodes were to be placed in the same rank, the value of
would be 0, and the result would be of little or no value in determining how important a node is. When all of the nodes receive a unique ranking, the value of
would be 1, and the ranking result would thus be perfectly monotonic.
Moreover, the complementary cumulative distribution function (CCDF) was utilized [
50], in addition to monotonicity, for a better evaluation of the ranking distribution of different methods. The value of the function is calculated for rank
r by Equation (31):
where
represents the number of nodes that were placed in rank
i, and
is the total number of network nodes. This function can better reflect the distribution of nodes to different ranks. Having more nodes gathered in a rank causes the function to sharply drop to zero, while having an extremely scattered distribution of nodes to different ranks makes the function have a mild slope. However, differentiation and monotonicity alone cannot be used to identify whether or not a ranking method is successful; hence, we have also tested the method in terms of its precision and the correctness of the resulting rankings.
In order to assess the precision of EMI, we compared the ranking list that was generated by EMI with those of the benchmarking methods using the real spreading influence model susceptible-infected (
SI) [
28]. In this model, each node belongs to one of two states: susceptible (
S) or infectious (
I). At first, all nodes are set to be in the state
S except for node
, which is selected to be the infected node. At each timestamp, the infected nodes infect their susceptible neighbors with an infection probability
. The number of infected nodes is regarded as the influence of node
when the epidemic process is finished. However, the infection probability is not constant in a weighted network. The feature that is contained on the directed edge has to be considered as well. According to Yan et al. [
51], the infection probability in directed and weighted networks is defined as
, in which susceptible node
i is infected through neighbor node
j.
α corresponds to a constant of a positive value,
is the weight of a directed boundary, and
denotes the maximum value among
. In this experiment, we adopted
as the infection probability in directed and weighted networks. We first used the EMI and other methods that were mentioned above to find the ten most critical nodes. Then, we selected different sets (different numbers of nodes, usually at the top of ranking list) as the seed nodes and made them the infected nodes. The number of infected nodes was regarded as the correctness of a method when the propagation process was over. To increase accuracy, we ran the process hundreds of times, and the mean value was considered to be the final result. Finally, time-efficiency is the last but not the least criteria that we need to evaluate. A shorter running time means that the program is fast and efficient.
4.3. Experimental Results
In the first experiment, other well-known centrality measures were applied to six real networks. The monotonicity value of the ranking list that was obtained using each of the approaches was calculated.
Table 5 shows the results. The greater the value of M is, the more clearly a method can distinguish which rank the node belongs to. Based on the theory and
Table 5, EMI obtained the maximum monotonicity values for all datasets except US-Air and Chicago; for these datasets, closeness centrality and bridging centrality obtained the greatest values (0.98480 and 0.93045, respectively), which were slightly higher than those of EMI. However, some methods performed poorly; for example, centroid, DMNC, and eccentricity performed poorly in most of the datasets, and even obtained a value of zero in particular datasets. It is remarkable that half of the methods had an average M value of less than 0.5. Plainly, EMI had outstanding performance as compared to these authoritative methods. The M values of EMI were vastly superior to those of most methods.
To further assess the efficiency of different measures in assigning distinct ranks to each node, we examined the node distribution in the next part of our experiment. To make the result more concise, we compared EMI with the top four measures (betweenness centrality, closeness centrality, bridging centrality, and stress centrality). Four datasets (Dutch college, US-Airport, Air traffic control, and Dolphins) were used in this part. As shown in
Figure 5, EMI did a perfect job in Dutch college and Dolphins, as every node was assigned to a distinct rank. Closeness centrality did not behaved very well in these two datasets. In US-Airport, EMI and closeness centrality performed the best compared with stress centrality and betweenness centrality, which placed a lot of nodes in the same rank. In the Air traffic control network, although EMI placed dozens of nodes in the same rank, the number was far lower than those of the other four methods. As shown in
Figure 5, stress centrality, betweenness centrality, and bridging centrality assigned almost 200 nodes to the same rank in the Air traffic control network. The ranking list that was obtained by closeness centrality had three ranks, into which were assigned 50, 100, and 150 nodes. Generally speaking, the ranking list that was obtained by EMI had a lower number of nodes in the different ranks, which is an urgent need when faced with networks containing millions of vertices.
In addition, the CCDF is plotted for Dutch college, US-Airport, E-road, and Dolphins in
Figure 6. Due to the large number of data points, we still chose, as we did before, to compare EMI with the top four measures. As mentioned earlier, the lower the number of nodes in the same rank, the slower will the curve decrease to zero. In Dutch college and Dolphins, EMI performed as well as most of the well-known measures and much better than closeness centrality. From the US-Airport and E-road datasets, we could see that EMI emerged as the best candidate as the size of the network increases. Closeness centrality and stress centrality simply indicate that a node was heavily involved in the core but was not relevant to maintaining communication between other nodes. Accordingly, bridging centrality and betweenness centrality focus on the shortest path, could not differentiate between nodes with different influences, and the nodes were distributed in less-distinct ranks. EMI outperformed the top four well-known methods; thus, it could be seen that deep disparities remain between EMI and the other 10 famous methods. Based on the above conclusion, it could be inferred that EMI distributed nodes to a larger number of ranks, and each rank contained a lower number of nodes. EMI was the best choice when a large dataset requires ranking. In the next experiment, we assessed the correctness and accuracy of all of the measures.
Experiments were performed to examine the methods’ accuracy by comparing the top 10 vital nodes.
Table 6 shows the lists of the top 10 key nodes produced by EMI and the other 10 approaches. In the Dutch college network, the proposed method, Bridging centrality, and DMNC had two and three nodes (red mark) in common. EMI is dissimilar to the other methods in terms of common nodes; however, five nodes is not a small number for a network with 32 vertices. In the US-Airport network, the methods almost have no resemblance to each other; EMI and DMNC were the only methods with half of the nodes in common. For the Air Traffic Control network, the number of the same nodes in the list between the proposed method and degree centrality, K-shell, closeness centrality, betweenness centrality, stress centrality, and radiality centrality was 8, 1, 6, 2, 4, and 6, respectively. EMI and four methods (closeness centrality, betweenness centrality, stress centrality, and radiality centrality) had only one node in common in the E-road network. In addition, the fact that the same nine nodes were identified by EMI and degree centrality is noteworthy, as it means that our method performs as well as degree centrality. Moreover, it can be seen that, in the Chicago and Dolphins networks, our proposed method and the other popular methods have almost all of the nodes in common. To sum up, we could observe that our proposed measure could produce acceptable results and provided accurate ranks for the top-N nodes in comparison with other methods. There was no conflict between EMI and other well-known methods.
As the datasets were ranked by all of the measures, we ran the SI simulation on the six datasets. In light of the size of the networks, we selected the top 2, 4, 6, 8, and 10 nodes to be the seed set in the Dutch college and Dolphins networks, and the top 10, 20, 30, 40, and 50 nodes to be the seed set in the US-Airport, Air traffic control, E-road, and Chicago networks. Correspondingly,
Figure 7 illustrates the results of the infection epidemic of each measure with different seed sets for the six networks. As shown in
Figure 7, in terms of all of the networks, centroid centrality, which is a shortest-path-based method, performed poorly. One possible reason for this is that centroid centrality, which only takes into consideration the information about the shortest path, does not employ the information that is carried by other paths. Eccentricity centrality and closeness centrality also performed poorly in many datasets, and their underlying principle may explain why. Interestingly, no method did well in all datasets except for EMI, even though stress centrality performed the best in US-Airport. As can be seen in
Figure 7, most methods were very unstable and their performance was mixed. Further, the single principle that they followed became ineffective when faced with increasingly complex and dynamic topological characteristics. In US-Airport, E-road, and Chicago, the top 10 most influential nodes as ranked by bridging centrality, centroid centrality, and closeness centrality had only infected dozens of nodes in a network with more than 1000 nodes, which illustrates that these methods had lost effectiveness and mistakenly made some isolated nodes the top key vertices.
Our proposed method EMI was the preferred method among the 11 methods and in the Dutch college, Air traffic control, E-road, Chicago, and Dolphins networks. However, stress centrality and some other methods had slightly better performance in seed sets 10, 20, 30, and 40. The infection epidemic of EMI increased as the seed set increased and caught up with stress centrality in seed set 50. Thus, we could deduce that EMI would perform the best if the seed set grew consistently. In addition, EMI’s infection epidemic in each seed set took first place in the Dutch college, Air traffic control, E-road, and Dolphins networks, which proved that our proposed method, EMI, could precisely identify which nodes were the most important as compared with other famous centrality methods. These datasets basically covered all of the types of complex networks. That is why most of the methods performed exactly the same: sometimes good, sometimes bad. Amazingly, EMI shows relatively stable performance in all datasets. Our proposed method solved this issue once and for all type of complex network.
Finally, the time consumption of each method was considered. We employed Cytoscape, an open source bioinformatics software platform, to run these methods and obtain the uptime. The results of the experiment are shown in
Table 7. The proposed method recorded a time of 0.015 s, 0.592 s, 0.039 s, 0.022 s, 0.018 s, and 0.0005 s in Dutch college, US-Airport, Air traffic control, E-road, Chicago, and Dolphins, respectively, which was far less than that of any other method. Therefore, EMI has emerged as the best choice when dealing with large datasets.