PageRank of Gluing Networks and Corresponding Markov Chains

Abstract

This paper studies Google’s PageRank algorithm. By an innovative application of the method of gluing Markov chains, we study the properties of Markov chains and extend their applicability by accounting for the damping factor and the personalization vector. Many properties of Markov chains related to spectrums and eigenvectors of the transition matrix, including the stationary distribution, periodicity, and persistent and transient states, will be investigated as well as part of the gluing process. Using the gluing formula, it is possible to decompose a large network into some sub-networks, compute their PageRank separably and glue them together. The computational workload can be reduced.

1. Introduction

In this section, we give a brief introduction to Markov chains and their applications. And Google PageRank, as an important application of Markov chains, is the main object with which we deal in this paper. Then we introduce our main ideas, such as gluing Markov chains to study Google PageRank, and show the advantage of these ideas compared to traditional methods.

1.1. Recent Results for Markov Chains

Markov chains were introduced by Andrey Markov in 1906 [1] to model systems transitioning between states with probabilities. Markov chains appear as the behind-the-scenes numerical structure in a huge amount of fields; thus, they have their application in a huge amount of areas, such as seismology, computer networks, epidemiology, and social dynamics.

Markov chains have been effectively applied in seismology to model earthquake occurrences and assess seismic hazards. Bountzis et al. [2] used the MAP method to analyze the temporal evolution of earthquakes in the Corinth Gulf, Greece, revealing high burstiness in the eastern subarea and demonstrating that MAPs outperform traditional models like the Gamma distribution. Votsi et al. [3] applied a discrete-time Markov model to the Hellenic Subduction Zone, estimating key indicators such as hitting times and failure rates and providing probabilistic seismic hazard results. Ünal et al. [4] combined k-means clustering with Markov chains to study earthquake sequencing in the Aegean Graben system of Turkey, identifying stationary distributions and mean passage times for different clusters. These studies highlight the versatility of Markov Chains in capturing the complex dynamics of earthquake occurrences and improving hazard predictions.

The application of Markov chains in modeling the spread of computer viruses within networks has been greatly explored in recent studies. Amador and Artalejo in [5] explore the stochastic SIRS model with warning signals, providing additional insights into the model’s behavior and characteristics. Then Amador [6] investigates a stochastic SEIQS model for computer viruses using continuous-time Markov chains, analyzing key characteristics such as stationary distributions, extinction and hazard times, and transient behaviors of susceptible, exposed, infectious, and quarantined computers. Papageorgiou and Vasiliadis [7] introduced a discrete-time non-homogeneous Markov system with state capacities, related to an SIQS model for computer networks. Their research highlights the importance of quarantine capacity and the impact of infection rates on network sustainability.

Markov chains are actively applied in epidemiology to improve the understanding and prediction of epidemic dynamics. Economou and Lopez–Herrero [8] studied the deterministic SIS model in a Markovian random environment, describing the influence of environmental changes on infection and recovery rates. Papageorgiou and Tsaklidis [9] further extended the stochastic SIRD model, focusing on the computation of stochastic properties such as the alert time and the infection and mortality times of tagged individuals. Papageorgiou [10] developed a stochastic SIRD model with imperfect immunity, investigating various stochastic properties like the extinction time of epidemics and the infection and mortality times of individuals. Papageorgiou et al. [11] proposed a novel method integrating the Extended Kalman Filter with recursive algorithms to estimate stochastic epidemic descriptors, such as the duration of outbreaks and the total number of infections and deaths. These studies collectively demonstrate the versatility and effectiveness of Markov chain models in analyzing epidemic phenomena and guiding public health interventions.

Markov chains are also useful to model social dynamics, studying how opinions or behaviors evolve through interactions in a network. They help analyze the impact of influence and network structure on collective behaviors. Kindermann R P, Snell J L. [12] gave a detailed introduction to this area. Recently, Bolzern et al. [13] showed that influence intensity affects opinion variance but not the mean under unbiased conditions. Similarly, Papageorgiou [14] used Markov chains to model the spread of radicalization, introducing new stochastic descriptors to better assess its severity.

1.2. Introduction for Google PageRank

Markov chains have wide applications, as the above shows. However, when it comes to one of the most influential and widespread applications of Markov chain, it is Google PageRank.

To understand how Markov chains appear on web pages, the CNN and Yahoo homepages are excellent examples. These pages are divided into distinct sections or branches, such as sports, economy, politics, health, etc. It is observed that there are numerous internal hyperlinks within each section, while the links connecting different sections are relatively small. Therefore, there should be a hidden algorithm designed to assign weights to each hyperlink, thereby determining their frequency of appearance.

Using this idea, Google founders Larry Page and Sergey Brin discovered a novel idea using graph theory, formulating the well-known Google PageRank [15]: A network can be seen as a directed graph. There should also be some numerical information on the edge to measure the importance or frequency of the corresponding link. In a word, these data form Markov chains over graphs [16].

More precisely, a network can be realized as a directed graph; more specifically, it is a graph with a finite vertex set, where each vertex represents a website and a directed edge from a to b if and only if there is a hyperlink from a to b. If there is a web surfer that starts from some page and goes to other pages in the same network via hyperlinks, this behavior is called a random walk on a directed graph. After a long time, it is hoped that the probability distribution converges to a limit depending on the initial distribution. The limiting distribution will be called the PageRank, and this value is usually used to rank the pages.

We will describe this by the following example:

Here, we study four websites and their hyperlinks.

• If the web surfer is at $1,$ it has 2 outgoing hyperlinks to 2 and $4,$ so in the next step the web surfer has probability ${\displaystyle \frac{1}{2}}$ to go to $2,$ and ${\displaystyle \frac{1}{2}}$ to go to $4.$
• Similarly, from 2, the web surfer has only 1 hyperlink to $1$, so there is a probability of 1 of going to 1, and so on...

In conclusion, if it has a limiting probability distribution, we can use $x_1,x_2,x_3,x_4$ to express their probability. Then it satisfies the following linear equations:

$$\begin{array}{c}{x}_2+{\displaystyle \frac{1}{3}}{x}_4=x_1\hfill \\ {\displaystyle \frac{1}{2}}{x}_1+{\displaystyle \frac{1}{2}}{x}_3+{\displaystyle \frac{1}{3}}{x}_4=x_2\hfill \\ {\displaystyle \frac{1}{3}}{x}_4=x_3\hfill \\ {\displaystyle \frac{1}{2}}{x}_1+{\displaystyle \frac{1}{2}}{x}_3=x_4,\hfill \end{array}$$

Solving this equation with the extra condition $x_1+x_2+x_3+x_4=1,$ we have the probability distribution

$$(x_1,x_2,x_3,x_4)=\left({\displaystyle \frac{5}{13}},{\displaystyle \frac{4}{13}},{\displaystyle \frac{1}{13}},{\displaystyle \frac{3}{13}}\right),$$

and this solution is unique. Then we have the rank of the pages $1>2>4>3.$

Remark 1. 

However, not all examples work using the algorithm above to yield a unique solution.

There are several good materials for the history and details of the Google PageRank, and we list some of them following. Rogers [17] explained with many examples how the weight can be designed to get the response faster. There are also many calculations and examples helping to understand how the PageRank works. Avrachenkov and Litvak [18] gave a more detailed discussion of the effect of newly created links and the extent to which a page can control its PageRank. They employed asymptotic analysis and mathematical modeling to study the impact of new links on Google PageRank. And they used Markov chain theory and matrix operations to derive conditions under which new links can increase the PageRank of a webpage and its neighbors, showing that linking within a relevant web community benefits PageRank, while irrelevant links can penalize the webpage and its community. Langville and Meyer [19] provided a comprehensive overview of the PageRank algorithm and its mathematical foundations. They used linear algebra, Markov chains, and numerical analysis to explain how PageRank is calculated and how it influences search engine rankings. Their book also shows many different algorithms that Google has applied for its research engines. Joshi and Patel [20] studied updates over time for the Google PageRank. They analyzed the impact of various factors such as backlinks, website age, content relevancy, traffic, keyword occurrence, and domain authority on PageRank through empirical observations and case studies. Their work shows that backlinks from high-authority websites significantly boost PageRank, while content quality and relevancy are crucial for maintaining high rankings. Frequent updates to the algorithm, such as Panda and Penguin, emphasize the importance of avoiding spammy practices and focusing on user experience.

1.3. Structure and Main Results of This Paper

In this paper, we introduce an approach to analyzing PageRank by employing the concept of gluing Markov chains. This method involves combining multiple Markov chains into a single chain by identifying certain states, allowing us to study the properties of the resulting chain and its relation to the original chains. This approach is particularly useful for understanding how the PageRank of a large network can be decomposed into smaller sub-networks, computed separately, and then recombined. This not only reduces the computational workload but also provides insights into the structural properties of the network.

The structure of this article is linear; we will first introduce the preliminaries, then establish the following results in order, and finally leave a section for discussion.

Our main results consist of the following parts:

After we introduce the concept of gluing Markov chains, we study the stationary distribution of the $\theta -$ gluing Markov chain Theorem 2.

Then we study the performance of some other properties of Markov chains after gluing, including the number of irreducible components Theorem 3 and the periodicity Theorem 4.

Lastly, we consider the network with the personalization vector and study how to glue the personalization vector as we glue the underlying Markov chains Theorem 7.

The idea of gluing Markov chains has its advantages compared to other results on computing PageRank; we provide some detailed discussion as follows.

Traditional methods for computing PageRank, as introduced by Page and Brin [15], rely on iterative algorithms such as the Power Iteration method. Their methods compute the PageRank values directly from the entire web graph, treating it as a single Markov chain. While effective, these methods can be computationally intensive, especially for large web graphs. For example, the power iteration method requires multiple iterations over the entire graph to converge to the stationary distribution, which can be time-consuming and resource-intensive. Our gluing approach decomposes the large network into smaller sub-networks, computes their PageRank values separately, and then recombines them. This reduces the computational complexity by allowing parallel processing of sub-networks and minimizing the number of iterations required for convergence. Additionally, our method provides a deeper understanding of the structural properties of the network by analyzing the interactions between sub-networks.

Personalized PageRank, introduced by Haveliwala [21], extends the traditional PageRank algorithm by incorporating user preferences through a personalization vector. This allows for more tailored search results and has been widely adopted in personalized search and recommendation systems. Personalized PageRank modifies the probability transition matrix by adding a damping factor and a personalization vector, ensuring convergence to a unique stationary distribution. While personalized PageRank focuses on modifying the transition matrix to incorporate user preferences, our gluing approach focuses on the structural decomposition of the network. Our method can be combined with personalized PageRank to further enhance the efficiency and relevance of search results. By decomposing the network, we can apply personalized PageRank to smaller sub-networks, reducing computational overhead and improving scalability.

Distributed computing approaches, such as those proposed by Kamvar et al. [22], leverage parallel processing to compute PageRank values more efficiently. These methods divide the web graph into smaller partitions and compute PageRank values in parallel across multiple processors or nodes. While these methods significantly reduce computation time, they require sophisticated distributed systems infrastructure and coordination mechanisms. Our gluing approach is inherently parallelizable, as it decomposes the network into smaller sub-networks that can be processed independently. The idea of decomposing networks in their approach is similar to ours. However, unlike distributed PageRank methods that require synchronization and communication between nodes, our method allows for independent computation of PageRank values for each sub-network. This reduces the complexity of distributed system design and improves the overall efficiency of the computation.

Approximate methods, such as those proposed by Andersen et al. [23], aim to reduce the computational complexity of PageRank by approximating the stationary distribution. These methods often use sampling techniques or truncated iterations to estimate PageRank values, trading off accuracy for computational efficiency. Our gluing approach provides an exact method for computing PageRank values by leveraging the structural properties of the network. While approximate methods can be faster, they may not always provide accurate results. Our method ensures precise computation of PageRank values by decomposing and recombining the results, maintaining accuracy while improving efficiency.

2. Preliminaries

In this section, we give the necessary preliminaries. Most of them are basic definitions and propositions on Markov chains.

2.1. Basic Definitions

We basically work with Markov chains and graphs. Now we give some explanation for their definitions.

Definition 1 

(Probability Distribution Vector and Stochastic Matrix ([24], Page 2)). A probability distribution vector of n states is a vector $\mathit{x}={(x_1,\cdots ,x_n)}^T$ where for $1\le i\le n$, we have $0\le x_i\le 1$ and ${\sum }_{i=1}^n{x}_i=1.$ An $n\times n$ matrix is called a stochastic matrix if its columns are probability distribution vectors.

Definition 2 

(Finite Markov Chain ([24], Page 2)). A discrete-time finite Markov chain $M=(S,\mathit{A})$ consists of the following data: Firstly, the set of states $S=\{x_1,\cdots x_n\}$, where the elements $x_i$ are functions from $\mathbb{N}$ to ${\mathbb{R}}_{\ge 0}$ such that the vector $x\left(t\right)={(x_1\left(t\right),\cdots ,x_n\left(t\right))}^T$ is a probability distribution vector of n states, $\forall t\in \mathbb{N}$; secondly, an $n\times n$ stochastic matrix $\mathit{A}$ such that $\mathit{x}(t+1)=\mathit{A}·\mathit{x}\left(t\right)$, $\forall t\in \mathbb{N}$.

The vector $\mathit{x}\left(0\right)$ is called the initial distribution, and $\mathit{A}$ is called the probability transition matrix of the Markov chain.

Each element $a_{i,j}$ of probability transition matrix $\mathit{A}$ reflects the probability of transition from $x_j$ to $x_i$. And the stochastic condition is necessary to ensure that $x_t$ are probability distributions for $t\ge 1$.

For a more generalized definition, see [25].

Now we fix some notation for general directed graphs $G=(V,E)$ ([26], Page 1). Usually, we will assume that there is no self-loop in the graph. There is an associated state vector $x$ corresponding to G. And we can construct a transition matrix $\mathit{A}$ for such a graph (for precise definitions, see [24], Page 8).

So we can construct a Markov chain from the graph, denoted by $M=(S,\mathit{A})$ (known as random walk on graphs, see [24], Page 8).

Definition 3 

(Stationary Distribution ([27], Page 21)). The stationary distribution (or equilibrium state) of a Markov chain is a probability distribution vector of n-states. $\mathit{x}$ satisfies $\mathit{x}=\mathit{Ax}$.

In the model of a network, we also call the corresponding stationary distribution the PageRank vector, meaning that each time we transfer importance, each page gives away the same importance it receives.

We also deal with undirected graphs. An undirected graph $G=(V,E)$ is a directed graph such that for any pair of vertices $i,j\in V$ the number of edges from i to j is the same as the number of j to i.

In the study of Markov chains, we would like to know the behavior of $\mathit{x}\left(t\right)$ when t gets large and how it depends on $\mathit{x}\left(0\right)$. There are also many examples and applications of Markov Chain other than PageRank.

Remark 2. 

There exists a Markov chain that has a unique stationary distribution, but the sequence of probability vector $\mathit{x}\left(t\right):={\mathit{A}}^t\mathit{x}\left(0\right)$ does not converge as $t\to +\infty$. For example, consider the Markov chain with $\mathit{x}\left(0\right)={(1,0,0,0)}^T$ and the transition matrix is of the following form:

$$\left[\begin{array}{cccc}0& 1& 0& 0\\ 0& 0& 1& 0\\ 1& 0& 0& 0\\ 0& 0& 0& 0\end{array}\right].$$

It is easy to verify the above claim for such a Markov chain.

2.2. Spectrum of $\mathit{A}$

Both the uniqueness of the stationary distribution and the convergence of $\mathit{x}\left(t\right)$ depend on the eigenvalues of $\mathit{A}$. In this section, we recall the related results. The most important is Perron–Frobenius theorem. It is a deep theorem leading to fruitful consequences; one can get more in [28].

Theorem 1 

(Perron-Frobenius [29]). For a matrix $\mathit{A}$ with all entries non-negative such that ${\mathit{A}}^m$ has all positive entries for some m, there exists a positive eigenvalue ${\lambda }_0$ of multiplicity one such that any absolute value of other eigenvalues is strictly less than ${\lambda }_0$. And there exists a left eigenvector of ${\lambda }_0$ that has all positive entries.

Proof. 

See [30] or Page 125 of [29]. □

For a general stochastic matrix, one can derive the following classical results.

Corollary 1 

([29], Page 126). For an $n\times n$ stochastic matrix $\mathit{A}$, let ${\lambda }_1,\cdots ,{\lambda }_n$ be all the eigenvalues of $\mathit{A}$; then we have the following properties:

(1)

The absolute value of eigenvalue $|{\lambda }_i|\le 1$.

(2)

The number 1 is an eigenvalue of the matrix.

(3)

For any eigenvalue satisfying $|{\lambda }_i|=1,$ then ${\lambda }_i$ is a root of unity. Especially when the elements in $\mathit{A}$ are all positive, then such ${\lambda }_i=1$ and the algebraic multiplicity of eigenvalue $\lambda =1$ is 1.

From the above corollary, we can also see that for general stochastic matrices, the algebraic multiplicity of eigenvalue $\lambda =1$ is the same as the geometric multiplicity.

Proposition 1. 

When all entries in stochastic matrix $\mathit{A}$ are positive, then ${\mathit{A}}^{k}x$ converges as k approaches infinity for any probability distribution vector $\mathit{x}$. The limit does not depend on the choice of $\mathit{x}$.

Note that the eigenvalue $\lambda =1$ has multiplicity 1, and other eigenvalues have norm strictly less than 1 by Theorem 1. The proposition follows from the Jordan canonical form and direct computations.

However, by the example given in Remark 2, this does not necessarily work when not every entry is positive. Motivated by this, we add the damping factor to make ${\mathit{A}}^k\mathit{x}$ converge.

2.3. Damping Factor

In PageRank, there is a modification of the Markov Chain by adding personal interests and a factor $0<d<1$ into the discussion to guarantee convergence. This happens for two reasons:

• In real life, the web surfer has probability $(1-d)$ to follow the hyperlinks to visit the next website and also probability d to jump to any of the pages according to the surfer’s personal interests. The probability for the random jump to the i-th website is denoted by $p_i$.
• It makes all entries of the transition matrix positive, guaranteeing convergence, which will be explained as follows.

Call the probability distribution vector $\mathit{p}={(p_1,\cdots ,p_n)}^T$ the personalization vector, which gives a weighting of which pages the web surfer is likely to randomly jump to and has all positive entries. Then let d be the damping factor. For each iteration, we have $\mathit{x}(t+1)=(1-d)\mathit{A}\mathit{x}\left(t\right)+d·\mathit{p}.$ Observe this converges as we end up with a matrix with fully positive entries.

$$\begin{array}{cc}\hfill \mathit{x}(t+1)& =\left[\right(1-d)\mathit{A}+d(\mathit{p}(1,\cdots ,1)\left)\right]\mathit{x}\left(t\right).\hfill \end{array}$$

Let $\mathit{C}=(1-d)\mathit{A}+d\mathit{p}(1,\cdots ,1)$. Then $\mathit{C}$ is a stochastic matrix, and all the entries of $\mathit{C}$ are positive. This ensures the convergence of $\mathit{x}\left(t\right)$ and the uniqueness of the limiting stationary distribution. The absolute value of the second eigenvalue has an upper bound of $1-d$, see [31]. This gives a convergence rate of $\mathit{x}\left(t\right)$. In real life, Google usually uses a damping factor of $0.15$ to achieve a good PageRank result for customers.

Remark 3. 

It is important to emphasize that the personalization vector $\mathit{p}$ is a probability distribution vector. On one hand, it reflects the probability for the random jump; thus, it should be automatically normalized. On the other hand, $\mathit{C}=\left[\right(1-d)\mathit{A}+d(\mathit{p}(1,\cdots ,1)\left)\right]\mathit{x}\left(t\right)$ being a stochastic matrix requires the condition that $\mathit{p}$ is a probability distribution vector.

2.4. Basic Properties of States

In this section, we recall some properties of states in Markov chain $M=(S,\mathit{A}).$

Definition 4 

(Closed; [25] Page 16). A subset T of S is called closed if it satisfies that for $j\in T$ and $i\in S\setminus T$, then the transition probability $a_{i,j}=0.$

A closed subset T can be viewed as a Markov chain by inheriting the original transformations $a_{i,j}$ from M.

Definition 5 

(Irreducible Closed Subset; [27] Page 39). Let $a_{i,j}^m$ be the $i,j$-th element (rather than a power of $a_{i,j}$) in ${\mathit{A}}^m:=\mathit{A}\times \cdots \times \mathit{A}$ (for m-times). Two states $i,j$ intercommunicate if $\exists m_1,m_2$ such that $a_{i,j}^{m_1}>0$ and $a_{j,i}^{m_2}>0$, where $a_{i,j}^m$ is the $(i,j)$-th element of ${\mathit{A}}^m$.

A closed subset $T\subset S$ in the Markov chain M is called irreducible if and only if all pairs of states in T intercommunicate.

When a transition matrix has eigenvalue 1 with multiplicity, it means the network can be split into multiple irreducible closed subsets. The dimension of the eigenspace would cause the existence of multiple linearly independent eigenvectors, which would be the PageRank vectors of the irreducible subsets. In particular, we have the following proposition.

Proposition 2. 

For any Markov chain, then there is a decomposition of $S=S_1\bigsqcup \cdots \bigsqcup S_k\bigsqcup S^{\prime }$ where each $S_j$ is an irreducible closed subset and $S^{\prime }$ is the set not containing any irreducible closed subsets. This splitting is unique.

Proof. 

See Page 126 of [29] □

Observe the decomposition yields a transition matrix of the form

$$\mathit{A}=\left[\begin{array}{ccccc}{\mathit{B}}_{\mathbf{1}}& 0& \cdots & 0& {\mathit{R}}_{\mathbf{1}}\\ 0& {\mathit{B}}_{\mathbf{2}}& \cdots & 0& {\mathit{R}}_{\mathbf{2}}\\ ⋮& ⋮& \ddots & ⋮& ⋮\\ 0& 0& \cdots & {\mathit{B}}_{\mathit{k}}& {\mathit{R}}_{\mathit{k}}\\ 0& 0& \cdots & 0& \mathit{Q}\end{array}\right],$$

where ${\mathit{B}}_i$ denotes the probability transition matrix of the Markov chain $M_i$, which corresponds to $S_i$. Each ${\mathit{B}}_i$ contributes exactly one multiplicity for eigenvalue 1. And $\mathit{Q}$ reflects the probability transition from $S^{\prime }$ to itself, and ${\mathit{R}}_{\mathit{i}}$ reflects the probability transition from $S^{\prime }$ to $S_i$.

Definition 6 

(Page 53 [27]).

• For each state, it is persistent if there is a probability 1 of a process starting from this state to eventually return to it.
• If a state is not persistent, it is transient. That is, if some importance leaves this state, it has a positive probability of not returning to it at all.

For instance, each state in a connected undirected graph is persistent.

In Proposition 2, all the states in $S^{\prime }$ are exactly the transient states. It is suggested to consult [32] to obtain more properties of states for readers who are interested.

3. Gluing of Markov Chains and Stationary Distributions

3.1. Definitions of Gluing

We would like to study the behavior of gluing graphs for two major reasons:

• In abstract graph theory, it is very natural to consider the gluing of graphs so that a complicated graph can sometimes be decomposed or cut into smaller pieces.
• It has real-world applications to sites that are hubs that lead to other disparate networks. Consider, for instance, a new website. From the main page, one can find many different hyperlinked pages—politics, sports, etc. However, these form disparate networks that might not hyperlink to each other at all. In that sense, the only reason they are not separate irreducible networks is due to the presence of a main hub that they intercommunicate with. If we see the main hub as a page in each of those networks that were glued together, we can calculate the individual PageRanks pre-gluing and then put them together to find the final PageRank, saving computational power. We can also do the same with personalization vectors and see how the local personalization vectors affect the global personalization vector.

First, we consider an example.

Example 1. 

Consider the gluing of two networks given by $1,2,3$ and $3,4$, respectively:

The first network has PageRank $\left({\displaystyle \frac{2}{9}},{\displaystyle \frac{4}{9}},{\displaystyle \frac{3}{9}}\right)$ and the second has $\left({\displaystyle \frac{1}{2}},{\displaystyle \frac{1}{2}}\right).$ The new network after gluing has PageRank $\left({\displaystyle \frac{2}{15}},{\displaystyle \frac{4}{15}},{\displaystyle \frac{6}{15}},{\displaystyle \frac{3}{15}}\right).$ The weighting is ${\displaystyle \frac{9}{15}}$ to ${\displaystyle \frac{6}{15}}.$ This leads us to consider whether we can develop the following proposition for gluing two Markov chains with equal weight, which is also the induction basis for the case of more general gluing processes:

Note that the networks and PageRank vectors here are actually Markov chains and their stationary distributions. Our arguments following fit for arbitrary Markov chains and their stationary distributions. So, from now on, we will replace networks and PageRank vectors by Markov chains and their stationary distributions.

We define the gluing of two graphs as follows.

Definition 7. 

For $i=1,2$, let $G_i=(V_i,E_i)$ be a directed graph with vertex set $V_i$ and edge $E_i$. Let $v_0^i$ be a fixed vertex in $V_i$. The gluing of $G_1$ and $G_2$ along $v_0^1$ and $v_0^2$ is a graph $G=(V,E)$ with vertex set

$$V=V_1\bigsqcup V_2/v_0^1\sim v_0^2,$$

which is the union of $V_1$ and $V_2$ quotient by the relation $v_0^1\sim v_0^2$. The quotient point is denoted by $v_0=v_0^1=v_0^2$. The edge set is $E=E_1\cup E_2$.

We have the following relations among probability transition matrices for random walks on these three graphs.

Proposition 3. 

Let $\mathit{B}={\left(b_{vw}\right)}_{v,w\in V_1}$ be the probability transition matrix for random walk on graph $G_1$ and $\mathit{C}={\left(c_{vw}\right)}_{v,w\in V_2}$ be the probability transition matrix for graph $G_2$. Then the probability transition matrix $\mathit{A}={\left(a_{vw}\right)}_{v,w\in V}$ for random walk on graph G has the following form:

$$a_{vw}=\left\{\begin{array}{cc}{b}_{vw}\hfill & \mathit{if}v\in V_1,w\in V_1\setminus \left\{v_0\right\}\hfill \\ c_{vw}\hfill & \mathit{if}v\in V_2,w\in V_2\setminus \left\{v_0\right\}\hfill \\ {\theta }_1{b}_{vw}\hfill & \mathit{if}v\in V_1\setminus \left\{v_0\right\},w=v_0\hfill \\ {\theta }_2{c}_{vw}\hfill & \mathit{if}v\in V_2\setminus \left\{v_0\right\},w=v_0\hfill \\ {\theta }_1{b}_{v_0{v}_0}+{\theta }_2{c}_{v_0{v}_0}\hfill & \mathit{if}v=w=v_0\hfill \\ 0\hfill & \mathit{otherwise}\hfill \end{array}\right.,$$

for some weights ${\theta }_1+{\theta }_2=1.$ More explicitly, let $g_i\left(v_0^i\right)$ be the number of outgoing edges from $v_0^i$ in $G_i$, here

$${\theta }_1={\displaystyle \frac{g_1\left(v_0^1\right)}{g_1\left(v_0^1\right)+g_2\left(v_0^2\right)}}\mathit{and}{\theta }_2={\displaystyle \frac{g_2\left(v_0^2\right)}{g_1\left(v_0^1\right)+g_2\left(v_0^2\right)}}.$$

Here, the weight ${\theta }_i$, $i=1,2$ can be seen as the probability of transition between two networks, which is accessible from the quotient point. Note that the gluing process can be generalized to fit for arbitrary finitely many graphs and their transition matrices. And note that a Markov chain is a combination of graphs and a more generalized probability transition matrix (that is, not the natural random walk transition matrix). So we can easily generalize to the concept of gluing finitely many Markov chains by analogue the process of gluing graphs and gluing transition matrix.

Definition 8 

(Gluing of Markov Chains). For integers $i=1,2,\cdots ,k$, let $M_i=(S_i,{\mathit{A}}_i)$ be k Markov chains, and $v_0^i$ be a fixed point in $S_i$. Assume $\mathit{\theta }=({\theta }_1,{\theta }_2,\cdots ,{\theta }_k),$ $0<{\theta }_i<1,$ and

$$\sum _{i=1}^k{\theta }_i=1.$$

The θ-gluing of $M_i$ along $v_0^i$ is the following Markov chain $M=(S,\mathit{A})$. The state set

$$S=\underset{i=1}{\overset{k}{⨆}}{S}_i/v_0^1\sim v_0^2\sim \cdots \sim v_0^k$$

is the disjoint union of $S_i$ quotient by relation $v_0^1\sim v_0^2\sim \cdots \sim v_0^k$ and denote this quotient point by $v_0=v_0^1=v_0^2=\cdots =v_0^k$. Let ${\mathit{A}}_i={\left(a_{i,vw}\right)}_{v,w\in S_i}$. Then the transition matrix $\mathit{A}$ is defined by

$$a_{vw}=\left\{\begin{array}{cc}{a}_{i,vw}\hfill & \mathit{if}v\in S_i,w\in S_i\setminus \left\{v_0\right\}\hfill \\ {\theta }_i{a}_{i,vw}\hfill & \mathit{if}v\in S_i\setminus \left\{v_0\right\},w=v_0\hfill \\ \sum _{i=1}^k{\theta }_i{a}_{i,v_0{v}_0}\hfill & \mathit{if}v=w=v_0\hfill \\ 0\hfill & \mathit{otherwise}\hfill \end{array}\right..$$

3.2. Theorem of Gluing Stationary Distributions

Proposition 4. 

Under the notation and assumption of the definition above, gluing two Markov chains $M_1=(S_1,{\mathit{A}}_{\mathbf{1}})$ and $M_2=(S_2,{\mathit{A}}_{\mathbf{2}})$ at vertices $v_0^1\in S_1$ and $v_0^2\in S_2$ with weight $({\theta }_1,{\theta }_2)$, we obtain a new Markov chain $M=(S,\mathit{A})$. Let $\mathit{y}={\left({\mathit{y}}_v\right)}_{v\in V_1},\mathit{z}={\left(z_v\right)}_{v\in V_2}$ be stationary distributions of $M_1,M_2$, respectively. Then there is a stationary distribution $\mathit{x}={\left(x_v\right)}_{v\in V}$ of the new glued Markov chain with the following stationary distribution

$$x_v=\left\{\begin{array}{cc}\alpha y_v\hfill & \mathit{if}v\in V_1\setminus \left\{v_0^1\right\}\hfill \\ \beta z_v\hfill & \mathit{if}v\in V_2\setminus \left\{v_0^2\right\}\hfill \\ \alpha y_{v_0^1}+\beta z_{v_0^2}\hfill & \mathit{if}v=v_0^1=v_0^2\hfill \end{array}\right.,$$

where

$$\begin{array}{cc}\hfill \alpha & ={\displaystyle \frac{{\theta }_1{z}_{v_0^2}}{{\theta }_1{z}_{v_0^2}+{\theta }_2{y}_{v_0^1}}}={\displaystyle \frac{{\theta }_1\frac{1}{y_{v_0^1}}}{{\theta }_1\frac{1}{y_{v_0^1}}+{\theta }_2\frac{1}{z_{v_0^2}}}}\hfill \\ \hfill \beta & ={\displaystyle \frac{{\theta }_2{y}_{v_0^1}}{{\theta }_1{z}_{v_0^2}+{\theta }_2{y}_{v_0^1}}}={\displaystyle \frac{{\theta }_2\frac{1}{z_{v_0^2}}}{{\theta }_1\frac{1}{y_{v_0^1}}+{\theta }_2\frac{1}{z_{v_0^2}}}}.\hfill \end{array}$$

Proof. 

Consider the stationary equation $\mathit{Ax}=\mathit{x}$. For each row, we have equation $x_v={\sum }_w{a}_{vw}{x}_w$. Now we have three cases for index v, $v\in V_1\setminus \left\{v_0^1\right\}$, $v\in V_2\setminus \left\{v_0^2\right\}$ and $v=v_0^1=v_0^2$.

In the first case, $v\in V_1\setminus \left\{v_0^1\right\}.$ Then we have

$$\begin{array}{cc}\hfill \alpha y_v& =x_v=\sum _{w\in x}{a}_{vw}{x}_w\hfill \\ \hfill & =\sum _{w\in V_1\setminus \left\{v_0^1\right\}}{a}_{vw}{x}_w+a_{v{v}_0^1}{x}_{v_0^1}+\sum _{w\in V_2\setminus \left\{v_0^2\right\}}{a}_{vw}{x}_w\hfill \\ \hfill & =\sum _{w\in V_1\setminus \left\{v_0^1\right\}}{b}_{vw}·\alpha y_w+a_{v{v}_0^1}{x}_{v_0^1}\hfill \\ \hfill & =\sum _{w\in V_1\setminus \left\{v_0^1\right\}}{b}_{vw}·\alpha y_w+{\theta }_1{b}_{v{v}_0}(\alpha y_{v_0^2}+\beta z_{v_0^2})\hfill \\ \hfill & =\alpha \left(\sum _{w\in V_1}{b}_{vw}·y_w\right)-(1-{\theta }_1)b_{v{v}_0}\alpha y_{v_0^2}+{\theta }_1{b}_{v{v}_0}\beta z_{v_0^2}\hfill \\ \hfill & =\alpha y_v-{\theta }_2{b}_{v{v}_0}\alpha y_{v_0^2}+{\theta }_1{b}_{v{v}_0}\beta z_{v_0^2},\hfill \end{array}$$

yielding

$$\begin{array}{cc}& {\theta }_2{b}_{v{v}_0}\alpha y_{v_0^2}={\theta }_1{b}_{v{v}_0}\beta z_{v_0^2}\hfill \\ & {\theta }_2\alpha y_{v_0^2}={\theta }_1\beta z_{v_0^2}\hfill \\ & {\displaystyle \frac{\alpha }{\beta }}={\displaystyle \frac{{\theta }_1{z}_{v_0^2}}{{\theta }_2{y}_{v_0^1}}}.\hfill \end{array}$$

Analogously, we can find the same $\alpha$ to $\beta$ ratio in the second case $v\in V_2\setminus \left\{v_0^2\right\}$. For the third case, take the $\alpha =\beta$ ratios given from the previous two parts of the ansatz. Then

$$\begin{array}{cc}& \alpha y_{v_0^1}+\beta z_{v_0^2}=\sum _{w\in V_1\setminus \left\{v_0^1\right\}}{\theta }_1{a}_{v_0^{1}w}·x_w+\sum _{w\in V_2\setminus \left\{v_0^2\right\}}{\theta }_2{a}_{v_0^{2}w}{x}_w\hfill \\ & \alpha \sum _{w\in V_1\setminus \left\{v_0^1\right\}}{a}_{v_0^{1}w}{y}_w+\beta \sum _{w\in V_2\setminus \left\{v_0^2\right\}}{a}_{v_0^{2}w}{z}_w=\sum _{w\in V_1\setminus \left\{v_0^1\right\}}{\theta }_1{a}_{v_0^{1}w}·x_w+\sum _{w\in V_2\setminus \left\{v_0^2\right\}}{\theta }_2{a}_{v_0^{2}w}{x}_w,\hfill \end{array}$$

for which the substitution $x_w=\alpha y_w$ for $w\in V_1\setminus \left\{v_0^1\right\}$ and $x_w=\beta z_w$ for $w\in V_2\setminus \left\{v_0^2\right\}$ works. □

The above proposition can be inductively generalized to the following theorem, which describes the stationary distribution after a general weighted gluing.

Theorem 2 

(Gluing Stationary Distributions). Let $M=(S,\mathit{A})$ be a θ-gluing of Markov chains $M_i$ in Definition 8. Let ${\mathit{x}}^{\mathit{i}}={\left(x_v^i\right)}_{v\in S_i}$ be a stationary distribution of $M_i$ and assume $x_{v_0}^i\ne 0$. Denote by

$${\alpha }_i:={\displaystyle \frac{{\theta }_i}{x_{v_0}^i}}\mathit{and}\alpha :=\sum _{i=1}^k{\alpha }_i.$$

Then there is a $\mathit{x}={\left(x_v\right)}_{v\in S}$ of M

$$x_v=\left\{\begin{array}{cc}{\displaystyle \frac{{\alpha }_i}{\alpha }}{x}_v^i\hfill & \mathrm{if}v\in S_i\setminus \left\{v_0^i\right\}\hfill \\ \sum _{i=1}^k{\displaystyle \frac{{\alpha }_i}{\alpha }}{x}_{v_0}^i\hfill & \mathrm{if}v=v_0\hfill \end{array}\right..$$

Proof. 

Suppose that this holds until the gluing of k Markov chains. Take the gluing of the $k+1$ Markov chains. First, we glue the k Markov chains together; we then glue them with the $(k+1)$ th network.

More explicitly, from $\mathit{\theta }=({\theta }_1,\cdots ,{\theta }_{k+1})$, we define a system of new weights $\tilde{\mathit{\theta }}=(\widetilde{{\theta }_1},\cdots ,\widetilde{{\theta }_k})$ with

$$\widetilde{{\theta }_i}={\displaystyle \frac{{\theta }_i}{{\sum }_{i=1}^k{\theta }_i}}$$

The $\tilde{\mathit{\theta }}$-gluing of $M_i=(S_i,{\mathit{A}}_i)$ gives a Markov chain $\tilde{M}$. Then we can see that the $(1-{\theta }_{k+1},{\theta }_{k+1})$ gluing of $\tilde{M}$ with $M_{k+1}$ is exactly the $\mathit{\theta }$-gluing of $M_1,\cdots ,M_{k+1}$.

The proof of Proposition 4 does not use the explicit form of $\mathit{\theta }$ in terms of outgoing edges. So it still works for a $\mathit{\theta }$-gluing of two Markov chains. So we obtain the required identity based on the gluing formula for equilibrium vectors of two Markov chains. □

Example 2. 

Let us use the following example to illustrate that the gluing method does save computing power.

We use the code provided by the following website: http://www.pagerank.dk/, to perform the calculations of the examples.

Consider the following two Markov chains coming from random walks on graphs with corresponding probability transition matrices of the following form, respectively:

$$\left[\begin{array}{cccc}{\displaystyle \frac{1}{3}}& {\displaystyle \frac{1}{2}}& 0& 0\\ {\displaystyle \frac{1}{3}}& 0& 1& {\displaystyle \frac{1}{2}}\\ {\displaystyle \frac{1}{3}}& {\displaystyle \frac{1}{2}}& 0& 0\\ 0& 0& 0& {\displaystyle \frac{1}{2}}\end{array}\right]\mathit{and}\left[\begin{array}{ccccc}0& 0& 0& 0& {\displaystyle \frac{1}{3}}\\ {\displaystyle \frac{1}{2}}& 0& 0& {\displaystyle \frac{1}{2}}& 0\\ 0& {\displaystyle \frac{1}{2}}& 1& {\displaystyle \frac{1}{2}}& {\displaystyle \frac{1}{3}}\\ {\displaystyle \frac{1}{2}}& {\displaystyle \frac{1}{2}}& 0& 0& 0\\ 0& 0& 0& 0& {\displaystyle \frac{1}{3}}\end{array}\right].$$

We glue the last state of the first Markov chain and the first state of the second Markov chain together and we use the gluing weight as Proposition 3. Then the gluing probability transition matrix is the following:

$$\left[\begin{array}{cccccccc}{\displaystyle \frac{1}{3}}& {\displaystyle \frac{1}{2}}& 0& 0& 0& 0& 0& 0\\ {\displaystyle \frac{1}{3}}& 0& 1& {\displaystyle \frac{1}{4}}& 0& 0& 0& 0\\ {\displaystyle \frac{1}{3}}& {\displaystyle \frac{1}{2}}& 0& 0& 0& 0& 0& 0\\ 0& 0& 0& {\displaystyle \frac{1}{4}}& 0& 0& 0& 0\\ 0& 0& 0& 0& 0& 0& 0& {\displaystyle \frac{1}{3}}\\ 0& 0& 0& {\displaystyle \frac{1}{4}}& 0& 0& {\displaystyle \frac{1}{2}}& 0\\ 0& 0& 0& 0& {\displaystyle \frac{1}{2}}& 1& {\displaystyle \frac{1}{2}}& {\displaystyle \frac{1}{3}}\\ 0& 0& 0& {\displaystyle \frac{1}{4}}& {\displaystyle \frac{1}{2}}& 0& 0& 0\\ 0& 0& 0& 0& 0& 0& 0& {\displaystyle \frac{1}{3}}\end{array}\right].$$

Firstly, we compute the stationary state of the gluing Markov chain directly. Secondly, we compute the stationary state for the two Markov chains and glue them together by the formula in Theorem 2. We compare the iterations at which the stationary distribution converges to three decimal places in above two cases.

The first case, that is, the classical method to compute the stationary distribution, spends 47 iterations, as the following table shows.

Classical Method
Iteration	0	10	20	30	40	50
state 1	0.125	0.052	0.014	0.004	0.001	0.000
state 2	0.125	0.038	0.010	0.003	0.001	0.000
state 3	0.125	0.014	0.004	0.001	0.000	0.000
state 4	0.125	0.038	0.009	0.003	0.001	0.000
state 5	0.125	0.000	0.000	0.000	0.000	0.000
state 6	0.125	0.000	0.000	0.000	0.000	0.000
state 7	0.125	0.000	0.000	0.000	0.000	0.000
state 8	0.125	0.858	0.962	0.990	0.997	1.000

While the second case, which is our gluing method, spends 14 iterations:

Gluing Method
Iteration	0	10	20	30	40	50
state 1	0.125	0.000	0.000	0.000	0.000	0.000
state 2	0.125	0.000	0.000	0.000	0.000	0.000
state 3	0.125	0.000	0.000	0.000	0.000	0.000
state 4	0.125	0.002	0.000	0.000	0.000	0.000
state 5	0.125	0.000	0.000	0.000	0.000	0.000
state 6	0.125	0.001	0.000	0.000	0.000	0.000
state 7	0.125	0.000	0.000	0.000	0.000	0.000
state 8	0.125	0.996	1.000	1.000	1.000	1.000

From the simple example, we see that the gluing method does improve the computational resource. We believe the advantage becomes more obvious as the gluing Markov chain becomes of larger and larger size.

3.3. Example of Undirected Graphs

Recall that an undirected graph $G=(V,E)$ is a directed graph such that for any pair of vertices $v,w\in V$, the number of edges from v to w is the same as the number from w to v. In this section, we consider the Markov chains corresponding to random walks on undirected graphs; that is, the probability transition matrices are of the following form:

$$\begin{array}{c}\hfill {\left(a_{vw}\right)}_{v,w\in V}={\left({\displaystyle \frac{k_{vw}}{g\left(w\right)}}\right)}_{v,w\in V}.\end{array}$$

Here $k_{vw}$ is the number of edges from vertices v to w. And $g\left(w\right)$ is the valence of w [33], which is the number of edges that start from w.

For undirected graphs $G=(V,E)$, there is a canonical stationary distribution given by the valence of each vertex as follows. Let e be the number of total edges in E. Then the vector

$$\mathit{x}={\left({\displaystyle \frac{g\left(v\right)}{e}}\right)}_{v\in V}$$

automatically satisfies the equilibrium or PageRank equation. This is a direct computation; see page 4 of [33] for the case that $k_{vw}=0\mathrm{or}1$. We call this the canonical stationary distribution or the valence PageRank. Since the valence of the vertex is additive under gluing, we have the following observation.

Lemma 1. 

For a glued graph G of two undirected graphs $G_1,G_2$ without loops, the addition of valence remains the valence for G.

In Theorem 2, we only allow 1 state in each Markov chain to be glued. But for two undirected graphs $G_1,G_2$, we can allow arbitrarily many pairs of states to be glued.

Actually, the key obstruction for gluing arbitrary states in the general case is that the $\alpha$ and $\beta$ in Proposition 4 is dependent on the choice of pair of states to be glued, where $\alpha$ and $\beta$ can be seen as the distribution of importance of these two Markov chains. So for arbitrarily many pairs of states, we do not know which pair to choose to determine $\alpha$ and $\beta$. However, when $G_1$ and $G_2$ are undirected graphs, as we will see, such $\alpha$ and $\beta$ are independent of the choice of gluing pair. Therefore, they can be glued for arbitrarily many pairs of points in a natural way.

Let P be the set of pairs of vertices $(v^1,v^2)$ such that $v^i\in V_i$, for $i=1,2$. We assume that for any $v\in V_1$, there exists at most 1 element $w\in V_2$ such that $(v,w)\in P$, and symmetrically, we make the same assumption for $V_2$. We glue $G_1$ and $G_2$ along P to get an undirected graph G. Let $L_i$ be the subset of $V_i$ such that $\forall v\in V_i$, v appears as the $i$-th coordinate of some element in P, for $i=1,2$. Let L be a subset of vertex set V of G such that $\forall v\in L$ comes from the gluing of some pairs in P.

We define the gluing probability transition matrix $\mathit{A}:={\left(a_{vw}\right)}_{v,w\in V}$ as follows:

$$a_{vw}=\left\{\begin{array}{cc}{\displaystyle \frac{k_{vw}}{g\left(w\right)}}\hfill & \mathrm{if}v\in V_i,w\in V_i\setminus L_i\hfill \\ {\theta }_i^w{\displaystyle \frac{k_{v{w}_i}}{g\left(w_i\right)}}\hfill & \mathrm{if}v\in V_i\setminus L_i,w\in L\hfill \\ {\theta }_1^w{\displaystyle \frac{k_{v_1{w}_1}}{g\left(w_1\right)}}+{\theta }_2^w{\displaystyle \frac{k_{v_2{w}_2}}{g\left(w_2\right)}}\hfill & \mathrm{if}v,w\in L\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array}\right.i=1,2.$$

Note that for each $w\in L$, there exists a unique pair $(w_1,w_2)\in P$ gluing to w, now ${\theta }_w^i:={\displaystyle \frac{g\left(v_i\right)}{g\left(v_1\right)+g\left(v_2\right)}}$, analogous to the weight construct in Definition 7. One can check that matrix $\mathit{A}$ is nothing but the probability transition matrix for random walk on G.

Now, analogous to Proposition 4, we compute the corresponding $\alpha$ and $\beta$ for each $({\theta }_w^1,{\theta }_w^2)$. Then they are independent of the choice of $w\in L$. Indeed, let $e_i$ be the number of edges of $G_i$. Then we have $\alpha ={\displaystyle \frac{e_1}{e_1+e_2}}$ and $\beta ={\displaystyle \frac{e_2}{e_1+e_2}}$. Then we claim that the following formula analogous to Proposition holds:

Proposition 5. 

For $G_1=(V_1,E_1)$ and $G_2=(V_2,E_2)$ be two undirected graphs.

Let P, $L_i$ for $i=1,2$, α, and β be defined as above, and let $G=(V,E)$ be the gluing undirected graph of $G_1$ and $G_2$ along P with L gluing points. And denote ${\mathit{x}}_i={\left({\displaystyle \frac{g\left(v_i\right)}{e_i}}\right)}_{v_i\in V_i}$ by the canonical stationary distribution of random walk on $G_i$.

If we define the natural gluing stochastic vector $\mathit{x}={\left(x_v\right)}_{v\in V}$ as follows:

$$x_v=\left\{\begin{array}{cc}\alpha {\displaystyle \frac{g\left(v\right)}{e_1}}\hfill & \mathit{if}v\in V_1\setminus L_1\hfill \\ \beta {\displaystyle \frac{g\left(v\right)}{e_2}}\hfill & \mathit{if}v\in V_2\setminus L_2\hfill \\ \alpha {\displaystyle \frac{g\left(v_1\right)}{e_1}}+\beta {\displaystyle \frac{g\left(v_2\right)}{e_2}}\hfill & \mathit{if}v\in L,\mathit{glued}\mathit{from}(v_1,v_2)\hfill \end{array}\right.,$$

then $\mathit{x}$ is the canonical stationary distribution for random walk on G.

Indeed, $\mathit{x}={\left(x_v\right)}_{v\in V}$ equals ${\left({\displaystyle \frac{\widetilde{g\left(v\right)}}{e_1+e_2}}\right)}_{v\in V}$, where $\widetilde{g\left(v\right)}$ is the valence of v in the gluing graph, which is the sum of valences of v in $G_1$ and $G_2$ by lemma 1 if v is one of the gluing states. Now, G is undirected again and $\mathit{A}$ is the probability transition matrix of the random walk on G, then $\mathit{x}$ is a canonical stationary distribution of G as we introduced before.

That is, Proposition can be generalized to arbitrary many gluings for undirected graphs.

The gluing graph of undirected graphs without loops remains undirected without loops. By induction based on the above argument, a similar result to Theorem holds for gluing arbitrarily many states for undirected graphs.

4. Other Properties of Gluing Markov Chains

4.1. Number of Irreducible Components

Theorem 3. 

When two Markov chains $M_1=(S_1,{\mathit{A}}_{\mathbf{1}})$ and $M_2=(S_2,{\mathit{A}}_{\mathbf{2}})$ with $k_1$ and $k_2$ irreducible closed subsets, respectively, are $\mathit{\theta }-$ glued to a Markov chain M, the number of irreducible closed subsets in the glued network is either $k_1+k_2$ or $k_1+k_2-1.$

Proof. 

Let $a\in S_1$ and $b\in S_2$ be glued together to $s_0$. Let $S_1$ be split into irreducible closed subsets $P_i$ for $1\le i\le k_1$ and $P^{\prime },$ while $M_2$ is split into ${\mathit{Q}}_i$ for $1\le i\le k_2$ and ${\mathit{Q}}^{\prime }.$

• If the glued points are part of $P_i$ and ${\mathit{Q}}_j$ for some $i,j$, then $P_i$ and ${\mathit{Q}}_j$ now form one irreducible and closed network, and all other irreducible and closed networks in $S^{\prime }$ are still irreducible and closed, as other networks in S do not intercommunicate with any point in $P_i$ and thus not any $T.$ This yields a total of $k_1+k_2-1$ closed irreducible networks.
• If the glued points are part of $P_i$ and ${\mathit{Q}}^{\prime }$ or $P^{\prime }$ and ${\mathit{Q}}_j$, then $P_i$ and ${\mathit{Q}}_j$ are still irreducible and closed networks, yielding $k_1+k_2.$
• If the glued points are part of $P^{\prime },{\mathit{Q}}^{\prime }$, then all closed irreducible networks are still closed and irreducible, and the glued point is not closed and irreducible, yielding $k_1+k_2.$

□

Proposition 6. 

For a finite Markov chain $M=(S,\mathit{A})$, the multiplicity of eigenvalue 1 in the transition matrix $\mathit{A}$ is the same as the number of irreducible closed subsets in S.

Proof. 

See Page 126 of [29]. □

Corollary 2. 

The multiplicity of eigenvalue 1 after the gluing above is either $k_1+k_2$ or $k_1+k_2-1.$

Proof. 

The multiplicity of eigenvalue 1 is the number of irreducible closed subsets, of which there are either $k_1+k_2$ or $k_1+k_2-1.$ □

4.2. Periodicity

Consider a gluing of graphs 1, 2, 3, 4 and 4, 5, 6, 7, 8, 9.

Before gluing, each graph appears to cycle 4 and 6 values, respectively, but when they are glued together, some importance starting at 4 can return after either 4 or 6 cycles but still must always return at a number that is a multiple of $gcd(4,6)=2.$ We can formalize this definition using periodicity:

Definition 9 

(Periodicity). Denote $a_{ij}^n$ as the $ij$-th entry in ${\mathit{A}}^n:=\mathit{A}\times \cdots \times \mathit{A}$, where $\mathit{A}$ is the transition matrix. We say a node j has period d if

• If $a_{jj}^n>0$ then $n=md$ for some constant $d,$
• d is the maximum possible integer with the above property.

The periods of all $j\in S$ for an irreducible Markov chain $M=(S,\mathit{A})$ are the same and called the period of M.

Theorem 4. 

When we glue points with periods $b,c$, respectively, the new period of the glued point (and thus the glued graph) is $gcd(b,c).$

Proof. 

The statement of periodicity tells us that if we take an infinitely small node of importance and distribute it across the Markov chain any way we want, it will always reach the gluing point after a number of moves that is a multiple of the period. Starting from $v_0,$ we can move to a new point in network $V_1,$ and the next time we reach $v_0$, we must have traveled m steps where $m\equiv -1moda$ by definition. The other option is to move to a new point in network $V_2,$ and the next time we reach $v_0$, we must have traveled n steps where $n\equiv -1moda.$ Take any path that takes this infinitely small node of importance and split it into “groups,” where the ith group denotes the number of steps needed to go from reaching $v_0$ the $i-1$-th time to the i-th time, and we start at 0 times. Summing these groups gives us the amount of steps needed, which is a linear combination of $a,b.$ This must be 0 mod $gcd(a,b)$, so $gcd(a,b)$ satisfies the first criterion for period. To satisfy the second, take configurations traveling in networks $V_1$ and $V_2$ only, respectively, to show that the period must divide both $a,b$. □

Applying this to the previous example, the period is $gcd(4,6)=2$. Based on the period of each state, we have the following corollary.

Corollary 3. 

The period of a θ-gluing of k irreducible Markov chains is the greatest common divisor of the periods of the k Markov chains.

Remark 4. 

In the case that $\mathit{gcd}(b,c)=1$, the gluing Markov chain M becomes aperiodic. Then any eigenvalue of M, whose absolute value is 1, is actually 1.

If M is further assumed to be irreducible. Then any eigenvalue is either 1, which has multiplicity 1, or a complex number with absolute value strictly less than 1. Then by direct computation the same as 1 for any initial distribution, the Markov chain convergence, and the limit is independent of choice of the initial distribution.

4.3. Common Eigenvalues and Spectrum

From the proof of gluing PageRank vectors in Theorem 4, we can also consider eigenvalues other than 1 for the gluing matrices.

Theorem 5. 

Use the notation from Definition 7 and the assumption in Proposition 3. Suppose matrix $\mathit{B}$ has eigenvalue λ and matrix $\mathit{C}$ has eigenvalue μ. Then if $\lambda =\mu$, then it is also an eigenvalue of $\mathit{A}.$

The proof is the same weighted combination of eigenvectors as Theorem 4. This is consistent with the result for periods of gluing states. We first need a theorem that relates eigenvalues with modulus one to periodicity.

Theorem 6 

([29]). Let $\mathit{A}$ be the transition matrix for an irreducible Markov chain with period d; then all the d-th roots of unity are eigenvalues of $\mathit{A}$ with algebraic multiplicity 1. These are all the eigenvalues of A with an absolute value of 1.

When we glue k irreducible Markov chains with periods $d_1,\cdots ,d_k$ together, then the common eigenvalues with absolute value 1 for these transition matrices are exactly d-th roots of unity, where d is the greatest common divisor of $d_1,\cdots ,d_k$. Based on these observations, we can make a conjecture about the spectrum of gluing Markov chains.

Conjecture 1. 

Glue Markov chains $M_i=(S_i,{\mathit{A}}_{\mathit{i}})$ to a Markov chain $M=(S,\mathit{A})$, then all the eigenvalues of $\mathit{A}$ lie in the convex hull of the eigenvalues of all ${\mathit{A}}_{\mathit{i}}$ on the complex plane.

5. Personalization Vector Gluing

In this section, we revert to using the terms “network” and “PageRank vector” instead of “Markov chain” and “stationary distribution” to be consistent with terms such as “personalization vector” and “damping factor.”

Now we discuss the PageRank of the gluing of networks together with the damping factor and personalization vector. In the real application of PageRank, the method of damping factors and personalization vectors are usually included to guarantee convergence and provide information based on the web surfer’s personal interests. In Section 2.3, we have seen that the Markov chain with a damping factor model has a strictly positive transition matrix $\tilde{\mathit{A}}$. When the two networks are glued together, we have two main differences from $\mathit{\theta }$-gluing of Markov chains. Firstly, the modified transition matrices $\tilde{\mathit{A}}=(1-d)\mathit{A}+d\mathit{p}(1,1,\cdots ,1)$ are not glued according to the $\mathit{\theta }$-gluing formula. Secondly, usual Markov chains may have many states such that the self-transition probabilities are zero (for example, random walk on a graph without loop), but $\tilde{\mathit{A}}$ also has positive diagonal entries.

We say the PageRank vector $\mathit{x}$ is for the website M with damping factor d and personalization vector $\mathit{p}$ if it is the solution of $\tilde{\mathit{A}}\mathit{x}=\mathit{x}$ with $\tilde{\mathit{A}}$ explained above.

Furthermore, we need a choice of personalization vector for the glued network based on the personalization vectors for both networks $M_1=(S_1,{\mathit{A}}_{\mathbf{1}})$ and $M_2=(S_2,{\mathit{A}}_{\mathbf{2}})$. Let ${\mathit{p}}^{\mathit{i}}={\left(p_v^i\right)}_{v\in V_i}$ be the personalization vector for $G_i$. Based on the gluing formula in Theorem 4, we propose a choice of personalization vectors in the following theorem. Let $\mathit{y}={\left(y_v\right)}_{v\in V_1}$ be the PageRank vector for $M_1$ with damping factor d and personalization vector ${\mathit{p}}^{\mathbf{1}}$, and let $\mathit{z}={\left(z_v\right)}_{v\in V_2}$ be the PageRank vector for $M_2$ with damping factor d and personalization vector ${\mathit{p}}^{\mathbf{2}}$. Under the same notation as Theorem 4, we have the following result.

Theorem 7. 

If the personalization vector is chosen to be $\mathit{p}={\left(p_v\right)}_{v\in V}$ and

$$p_v=\left\{\begin{array}{cc}\alpha p_v^1\hfill & \mathit{if}v\in V_1,v\ne v_0^1\hfill \\ \beta p_v^2\hfill & \mathit{if}v\in V_2,v\ne v_0^2\hfill \\ \alpha p_{v_0}^1+\beta p_{v_0}^2\hfill & \mathit{if}v=v_0=v_0^1=v_0^2\hfill \end{array}\right.,$$

with the same $\alpha ,\beta$ as Proposition 4. Then there is a PageRank vector $\mathit{x}={\left(x_v\right)}_{v\in V}$ for the glued website $M=(S,\mathit{A})$ with damping factor d and personalization vector $\mathit{p}$ as follows:

$$x_v=\left\{\begin{array}{cc}\alpha y_v\hfill & \mathit{if}v\in V_1\setminus \left\{v_0^1\right\}\hfill \\ \beta z_v\hfill & \mathit{if}v\in V_2\setminus \left\{v_0^2\right\}\hfill \\ \alpha y_{v_0}+\beta z_{v_0}\hfill & \mathit{if}v=v_0=v_0^1=v_0^2\hfill \end{array}\right..$$

Proof. 

We need to verify the equilibrium

$$x_v=d\sum _{j\in V}{a}_{vj}{x}_j+(1-d)p_v.$$

(1)

Take the ansatz

$$x_v=\left\{\begin{array}{cc}\alpha y_i\hfill & \mathrm{if}v\in V_1,v\ne v_0^1\hfill \\ \beta z_i\hfill & \mathrm{if}v\in V_2,v\ne v_0^2\hfill \\ \alpha y_{v_0}+\beta z_{v_0}\hfill & \mathrm{if}v=v_0^1=v_0^2\hfill \end{array}\right.,$$

we know

$$\begin{array}{cc}& \mathit{y}=d\mathit{By}+(1-d){\mathit{p}}^{\mathbf{1}}\hfill \\ & \mathit{z}=d\mathit{Cz}+(1-d){\mathit{p}}^{\mathbf{2}}.\hfill \\ \hfill \mathbf{or}& y_v=d\sum _{j\in V_1}{b}_{vj}{y}_j+(1-d)p_v^1\hfill \\ & z_v=d\sum _{j\in V_2}{c}_{vj}{z}_j+(1-d)p_v^2.\hfill \end{array}$$

Now, we would like to prove Equation (1) for all $v\in V$. If $v\in V_1$ and $v\ne v_0^1,$ we are looking for

$$\begin{array}{cc}& \alpha y_v=d\sum _{j\in V_1,j\ne v_0}{a}_{vj}\alpha y_j+d·b_{v{v}_0}{\theta }_1(\alpha y_{v_0^1}+\beta z_{v_0^2})+(1-d)p_v\hfill \\ & \alpha \left(y_v-d\sum _{j\in V_1,j\ne h}{a}_{vj}{y}_j\right)=d·b_{v{v}_0^1}{\theta }_1(\alpha y_{v_0^1}+\beta z_{v_0^2})+(1-d)p_v\hfill \\ & \alpha \left(y_v-d\sum _{j\in V_1,j\ne h}{b}_{vj}{y}_j\right)=d·b_{v{v}_0^1}{\theta }_1(\alpha y_{v_0^1}+\beta z_{v_0^2})+(1-d)p_v,\hfill \end{array}$$

and we have

$$\begin{array}{cc}& y_v=d\sum _{j\in V_1}{b}_{vj}{y}_j+(1-d)p_v^1\hfill \\ & y_v=d\sum _{v\in V_{1}j\ne h}{b}_{vj}{y}_j+d{b}_{v{v}_0^1}{y}_{v_0^1}+(1-d)p_v^1\hfill \\ & y_v-d\sum _{j\in V_1,j\ne h}{b}_{vj}{y}_j=d{b}_{v{v}_0^1}{y}_{v_0^1}+(1-d)p_v^1\hfill \\ & \alpha \left(y_v-d\sum _{j\in V_1,j\ne h}{b}_{vj}{y}_j\right)=\alpha (d{b}_{v{v}_0^1}{y}_{v_0^1}+(1-d)p_v^1).\hfill \end{array}$$

Therefore, it remains to prove

$$\begin{array}{cc}& \alpha d{b}_{v{v}_0^1}{y}_{v_0^1}+(1-d)p_v^1=d·b_{v{v}_0^1}{\theta }_1(\alpha y_{v_0^1}+\beta z_{v_0^2})+(1-d)p_v\hfill \\ & \alpha (1-d)p_v^1-(1-d)p_v=d·b_{v{v}_0^1}{\theta }_1(\alpha y_{v_0^1}+\beta z_{v_0^2})-\alpha (d{b}_{v{v}_0^1}{y}_{v_0^1})\hfill \\ & \alpha (1-d)p_v^1-(1-d)p_v=d·b_{v{v}_0^1}(\alpha ({\theta }_1-1)y_{v_0^1}+\beta {\theta }_1{z}_{v_0^2})\hfill \\ & \alpha (1-d)p_v^1-(1-d)p_v=d·b_{v{v}_0^1}\left(-\alpha {\theta }_2{y}_{v_0^1}+\beta {\theta }_1{z}_{v_0^2}\right).\hfill \end{array}$$

If $v_0^1$ does not link to v, then $b_{v{v}_0^1}=0$, leaving $p_v=\alpha p_v^1.$ If $v_0^1$ does link to v and $p_v=\alpha p_v$, then the LHS is 0, and we have

$$\begin{array}{cc}& d(\alpha {\theta }_2{y}_{v_0^1})=d(\beta {\theta }_1{z}_{v_0^2})\hfill \\ & \alpha {\theta }_2{y}_{v_0^1}=\beta {\theta }_1{z}_{v_0^2}\hfill \\ & {\displaystyle \frac{\alpha }{\beta }}={\displaystyle \frac{{\theta }_1}{{\theta }_2}}·{\displaystyle \frac{z_{v_0^2}}{y_{v_0^1}}}.\hfill \end{array}$$

Next, suppose $v\in V_2$ and $v\ne v_0^2.$ This gives $p_v=\beta p_v^2$ and the same ${\displaystyle \frac{\alpha }{\beta }}$ ratio as earlier. Finally, suppose $v=v_0=v_0^1=v_0^2.$ Then we have $x_{v_0^1}=x_{v_0^2}=\alpha y_{v_0^1}+\beta z_{v_0^2}.$ We would like to have

$$x_{v_0^1}=d\sum _{j\in V_1\mathrm{and}{V}_2}{a}_{v_0^{1}j}+(1-d)p_{v_0^1},$$

or splitting this into two graphs,

$$x_{v_0^1}=d\left(\sum _{j\in V_1}\alpha b_{v_0^{1}j}{y}_j+\sum _{j\in V_2}\beta c_{v_0^{2}j}{z}_j\right)+(1-d)p_{v_0^1}.$$

Simplifying, the desired equality to be proved is

$$\alpha (1-d)p_{v_0^2}^1+\beta (1-d)p_{v_0^2}^2=(1-d)p_{v_0^1},$$

or

$$v_{v_0^1}=\alpha p_{v_0^1}^1+\beta p_{v_0^2}^2.$$

Now we may take:

$$\begin{array}{cc}\hfill {\displaystyle \frac{\alpha }{\beta }}& ={\displaystyle \frac{{\theta }_1}{{\theta }_2}}·{\displaystyle \frac{z_{v_0^2}}{y_{v_0^1}}}\hfill \end{array}$$

and let

$$\begin{array}{cc}\hfill p_v& =\left\{\begin{array}{cc}\alpha p_v^1\hfill & \mathrm{if}v\in V_1,v\ne v_0^1\hfill \\ \beta p_v^2\hfill & \mathrm{if}v\in V_2,v\ne v_0^2\hfill \\ \alpha p_{v_0^1}^1+\beta p_{v_0^2}^2\hfill & \mathrm{if}i=v_0^1=v_0^2.\hfill \end{array}\right.\hfill \end{array}$$

We need $\sum v_i=1,$ while we already know $\sum p_v^1=\sum p_v^2=1.$ This implies $\alpha +\beta =1.$ As ${\displaystyle \frac{\alpha }{\beta }}={\displaystyle \frac{{\theta }_1}{{\theta }_2}}·{\displaystyle \frac{z_{v_0^2}}{y_{v_0^1}}},$ we have:

$$\begin{array}{cc}\hfill \alpha & ={\displaystyle \frac{{\theta }_1{z}_{v_0^2}}{{\theta }_1{z}_{v_0^2}+{\theta }_2{y}_{v_0^1}}}\hfill \\ \hfill \beta & ={\displaystyle \frac{{\theta }_2{y}_{v_0^1}}{{\theta }_1{z}_{v_0^2}+{\theta }_2{y}_{v_0^1}}}.\hfill \end{array}$$

□

Theorem 7 proposes a method of gluing personalization vectors. It would be useful to test this proposal in real life. For instance, we could investigate the gluing of personalization vectors for a web surfer on different branches of a news website and test whether the gluing gives a good PageRank recommendation or matches with the usual total personalization vector.

6. Discussion and Future Directions

In this paper, we mainly studied the properties of PageRank vectors when networks are glued together at some vertices. We further generalized this concept to glue finitely many Markov chains and studied various properties of Markov chains under the gluing construction. Those properties are closely related to the eigenvalues of transition matrices.

The novelty of this paper lies in the application of the “gluing” method to Markov chains, specifically in the context of the PageRank formula in Theorem 2. By merging multiple Markov chains into a single chain through the identification of certain states, we are able to study the properties of the resulting chain and its relation to the original chains. This approach allows for the decomposition of a large network into smaller sub-networks, which can be analyzed separately and then recombined. This significantly reduces computational workload by enabling parallel processing and minimizing iterations required for convergence. Actually, from Example 2, one can see the gluing method is about 4 times faster than the classical method even in a very simple case. On the other hand, the gluing method also provides deeper insights into the structural properties of the network. The gluing method also has good compatibility with damping factors and personalization vectors, further enhancing its applicability and efficiency in real-world applications such as search algorithms on the web.

The limitations of this paper primarily stem from the complexities and uncertainties associated with gluing arbitrarily many states in Markov chains. We only establish the gluing formula for gluing 1 state for each Markov chain Theorem 2. However, the PageRank formula for gluing arbitrarily many states is not straightforward to establish. This is because the relative importance for stationary distributions of different Markov chains in the gluing process (such as $\alpha$ and $\beta$ in Proposition 3.5) is dependent on the choice of gluing states. When multiple states are involved, it becomes unclear which specific pair of states should be used to determine these coefficients. Moreover, the relationships between the glued states themselves add another layer of complexity, making the situation difficult to manage and analyze. However, as demonstrated in the paper, for random walks on undirected graphs, the PageRank formula holds true even when gluing an arbitrary number of states Section 3.3. This highlights a specific case where the challenges of multiple states gluing are successfully addressed, but it also underscores the broader difficulties encountered in more general situations.

There are still many directions we would like to explore in the future about the gluing of Markov chains. For example, the hitting time or cover time represents how long a web surfer needs to reach a certain website or cover all the network. It would be interesting to know how those quantities change under the $\theta$-gluing model. While it would be hard to find the exact formula similar to our stationary distributions, it is still possible to find some bounds or estimates in terms of the hitting time or cover time before gluing. For instance, see ([24], Proposition 10.29) for gluing two identical graphs at a common vertex.

We also would like to investigate more applications of $\theta$-gluing of Markov chains in real life other than PageRank, such as genetics (see [34], Chapter 7.4) or learning theory based on Markov chains (see [34] Chapter 7.5).