**Proof.**

*i*. We define the indicator variable *X* ˆ (*w*) *<sup>n</sup>*,*k*as follows.

$$
\mathcal{X}\_{n,k}^{(w)} = \begin{cases} 1 & \text{if } w \text{ is a prefix of at least one string in } P \\ 0 & \text{otherwise.} \end{cases}
$$

For each *X* ˆ (*w*) *<sup>n</sup>*,*k* , we have

$$\begin{split} \mathbb{E}[\hat{X}\_{n,k}^{(w)}] &= \mathbb{P}(\hat{X}\_{n,k}^{(w)} = 1) \\ &= 1 - P(\hat{X}\_{n,k}^{(w)} = 0) \\ &= 1 - (1 - \mathbf{P}(w))^n. \end{split} \tag{25}$$

Summing over all words *w* of length *k*, determines the generating function *H*ˆ (*z*):

$$\begin{split}H(z) &= \sum\_{n\geq 0} \mathbb{E}[\hat{X}\_{n,k}] z^n\\ &= \sum\_{w \in \mathcal{A}^k} \left(\frac{1}{1-z} - \frac{1}{1 - (1 - \mathbf{P}(w))z}\right). \end{split} \tag{26}$$

*ii*. Similar to in (18) and (20), we obtain

$$\begin{split} \mathbb{E}[(\hat{X}\_{n,k})\_2] &= \sum\_{\substack{w,w' \in \mathcal{A}^k \\ w \neq w'}} \mathbb{E}[\hat{X}\_{n,k}^{(w)} \hat{X}\_{n,k}^{(w')}] \\ &= \sum\_{\substack{w,w' \in \mathcal{A}^k \\ w \neq w'}} \left(1 - (1 - \mathbb{P}(w))^n - (1 - \mathbb{P}(w'))^n + (1 - \mathbb{P}(w) - \mathbb{P}(w'))^n\right). \end{split} \tag{27}$$

Subsequently, we obtain the generating function below.

$$\begin{split} \hat{\mathbf{C}}(\hat{\mathbf{z}}) &= \sum\_{n\geq 0} \mathbb{E}[(\hat{X}\_{n,k})\_2] z^n \\ &= \sum\_{\substack{w,w' \in \mathcal{A}^1 \\ w \neq w'}} \sum\_{n\geq 0} \left( 1 - (1 - \mathbf{P}(w))^n - (1 - \mathbf{P}(w'))^n + (1 - \mathbf{P}(w) - \mathbf{P}(w'))^n \right) z^n \\ &= \sum\_{\substack{w,w' \in \mathcal{A}^1 \\ w \neq w'}} \left( \frac{1}{1 - z} - \frac{1}{1 - (1 - \mathbf{P}(w))z} - \frac{1}{1 - (1 - \mathbf{P}(w'))z} \right) \\ &\quad + \sum\_{\substack{w,w' \in \mathcal{A}^1 \\ w \neq w'}} \frac{1}{1 - (1 - \mathbf{P}(w) - \mathbf{P}(w'))z}. \tag{28} \end{split} \tag{29}$$

Our first goal is to compare the coefficients of the generating functions in the two models. The coefficients are expected to be asymptotically equivalent in the desired range for *k*. To compare the coefficients, we need more information on the analytic properties of these generating functions. This will be discussed in Section 3.3.

#### *3.3. Analytic Properties of the Generating Functions*

Here, we turn our attention to the smallest singularities of the two generating functions given in Lemma 3. It has been shown by Jacquet and Szpankowski [21] that *Dw*(*z*) has exactly one root in the disk |*z*| ≤ *ρ*. Following the notations in [21], we denote the root within the disk |*z*| ≤ *ρ* of *Dw*(*z*) by *Aw*, and by bootstrapping we obtain

$$A\_w = 1 + \frac{1}{S\_w(1)} \mathbf{P}(w) + O\left(\mathbf{P}(w)^2\right). \tag{29}$$

We also denote the derivative of *Dw*(*z*) at the root *Aw*, by *Bw*, and we obtain

$$B\_{\mathcal{W}} = -S\_{\mathcal{W}}(1) + \left(k - \frac{2S\_{\mathcal{w}}'(1)}{S\_{\mathcal{w}}(1)}\mathbf{P}(\mathcal{w})\right) + O\left(\mathbf{P}(\mathcal{w})^2\right). \tag{30}$$

In this paper, we will prove a similar result for the polynomial *Dw*,*w*(*z*) through the following work.

**Lemma 5.** *If w and w are two distinct binary words of length k and δ* = √*p, there exists ρ* > 1*, such that ρδ* < 1 *and*

$$\sum\_{w \in \mathcal{A}^k} \lceil \| S\_{w, w'}(\rho) \| \le (\rho \delta)^k \theta \rceil \mathbf{P}(w) \ge 1 - \theta \delta^k. \tag{31}$$

**Proof.** If the minimal degree of *Sw*,*w*(*z*) is greater than > *k*/2, then

$$|S\_{w, \nu'}(\rho)| \le (\rho \delta)^k \theta. \tag{32}$$

for *θ* = (1 − *p*)−1. For a fixed *<sup>w</sup>*, we have

$$\begin{split} \sum\_{w \in \mathcal{A}^{k}} & \left[ S\_{w,w'}(z) \text{ has minimal degree } \leq \lfloor k/2 \rfloor \right] \mathbb{P}(w) \\ &= \sum\_{i=1}^{\lfloor k/2 \rfloor} \sum\_{w \in \mathcal{A}^{k}} \left[ \mathbb{S}\_{w,w'}(z) \text{ has minimal degree } = i \right] \mathbb{P}(w) \\ &= \sum\_{i=1}^{\lfloor k/2 \rfloor} \sum\_{w\_{1}, w\_{2} \in \mathcal{A}^{k}} \mathbb{P}(w\_{1} \dots w\_{i}) \\ &= \sum\_{w\_{i+1}, \ldots, w\_{k} \in \mathcal{A}^{k-i}} \left[ \mathbb{S}\_{w,w'}(z) \text{ has minimal degree } = i \right] \mathbb{P}(w\_{i+1} \dots w\_{k}) \\ &\leq \sum\_{i=1}^{\lfloor k/2 \rfloor} \sum\_{w\_{1}, w\_{2} \in \mathcal{A}^{k}} \mathbb{P}(w\_{i+1} \dots w\_{k}) p^{k-i} \\ &= \sum\_{i=1}^{\lfloor k/2 \rfloor} p^{k-i} \sum\_{w\_{1}, w\_{2} \in \mathcal{A}^{k}} \mathbb{P}(w\_{1} \dots w\_{i}) \\ &= \sum\_{i=1}^{\lfloor k/2 \rfloor} p^{k-i} \leq \frac{p^{k-\lfloor k/2 \rfloor}}{1-p}. \end{split} \tag{33}$$

This leads to the following

$$\sum\_{w \in \mathcal{A}^k} \mathbb{I} \text{ every term of } S\_{w, w'}(z) \text{ is of degree } > \lfloor k/2 \rfloor \lfloor \mathbf{P}(w)$$

$$= 1 - \sum\_{w \in \mathcal{A}^k} \mathbb{I} S\_{w, w'}(z) \text{ has a term of degree} \le \lfloor k/2 \rfloor \lfloor \mathbf{P}(w)$$

$$\ge 1 - \frac{p^{\lceil k/2 \rceil}}{1 - p} \ge 1 - \theta \delta^k. \tag{34}$$

**Lemma 6.** *There exist K* > 0*, and ρ* > 1 *such that pρ* < 1*, and such that, for every pair of distinct words w, and w of length k* ≥ *K, and for* |*z*| ≤ *ρ, we have*

$$|S\_{\mathbf{w}}(z)S\_{\mathbf{w}'}(z) - S\_{\mathbf{w},\mathbf{w}'}(z)S\_{\mathbf{w}',\mathbf{w}}(z)| \, > \, 0. \tag{35}$$

*In other words, Sw*(*z*)*Sw*(*z*) − *Sw*,*w*(*z*)*Sw*,*<sup>w</sup>*(*z*) *does not have any roots in* |*z*| ≤ *ρ.*

**Proof.** There are three cases to consider:

Case *i*. When either *Sw*(*z*) = 1 or *Sw*(*z*) = 1, then every term of *Sw*,*w*(*z*)*Sw*,*<sup>w</sup>*(*z*) has degree *k* or larger, and therefore

$$|S\_{w,w'}(z)S\_{w',w}(z)| \le k \frac{(p\rho)^k}{1 - p\rho}.\tag{36}$$

There exists *K*1 > 0, such that for *k* > *K*1, we have lim*k*→∞ *k* (*pρ*)*<sup>k</sup>* 1 − *pρ* = 0. This yields

$$\begin{split} |S\_{\mathbf{w}}(z)S\_{\mathbf{w}'}(z) - S\_{\mathbf{w},\mathbf{w}'}(z)S\_{\mathbf{w}',\mathbf{w}}(z)| &\geq |S\_{\mathbf{w}}(z)S\_{\mathbf{w}'}(z)| - |S\_{\mathbf{w},\mathbf{w}'}(z)S\_{\mathbf{w}',\mathbf{w}}(z)| \\ &\geq 1 - k \frac{(p\rho)^k}{1 - p\rho} > 0. \end{split} \tag{37}$$

Case *ii*. If the minimal degree for *Sw*(*z*) − 1 or *Sw*(*z*) − 1 is greater than *k*/2, then every term of *Sw*,*w*(*z*)*Sw*,*<sup>w</sup>*(*z*) has degree at least *k*/2. We also note that, by Lemma 9, |*Sw*(*z*)*Sw*(*z*)| > 0. Therefore, there exists *K*2 > 0, such that

$$\begin{split} |S\_{\mathfrak{w}}(z)S\_{\mathfrak{w}'}(z) - S\_{\mathfrak{w}',\mathfrak{w}}(z)S\_{\mathfrak{w},\mathfrak{w}'}(z)| &\geq |S\_{\mathfrak{w}}(z)S\_{\mathfrak{w}'}(z)| - |S\_{\mathfrak{w}',\mathfrak{w}}(z)S\_{\mathfrak{w},\mathfrak{w}'}(z)| \\ &> 0 \quad \text{for } k > K\_2. \end{split} \tag{38}$$

Case *iii*. The only remaining case is where the minimal degree for *Sw*(*z*) − 1 and *Sw*(*z*) − 1 are both less than or equal to *k*/2. If *w* = *w*1...*wk*, then *w* = *uw*1...*wk*−*m*, where *u* is a word of length *m* ≥ 1. Then we have

$$S\_{w',w}(z) = \mathbf{P}(w\_{k-m+1}...w\_k)z^m \left( S\_w(z) - O\left( (pz)^{k-m} \right) \right). \tag{39}$$

There exists *K*3 > 0, such that

$$\begin{split} |S\_{w',w}(z)| &\leq (p\rho)^m \left( |S\_{\mathfrak{w}}(z)| + O\left( (p\rho)^{k-m} \right) \right) \\ &= (p\rho)^m |S\_{\mathfrak{w}}(z)| + O\left( (p\rho)^k \right) \\ &< |S\_{\mathfrak{w}}(z)| \quad \text{for } k > K\_3 \,. \end{split} \tag{40}$$

Similarly, we can show that there exists *<sup>K</sup>*3, such that |*Sw*,*w*(*z*)| < |*Sw*(*z*)|. Therefore, for *k* > *K*3 we have

$$\begin{split} |\mathcal{S}\_{\mathbf{w}}(\mathbf{z})\mathcal{S}\_{\mathbf{w}'}(\mathbf{z}) - \mathcal{S}\_{\mathbf{w},\mathbf{w}'}(\mathbf{z})\mathcal{S}\_{\mathbf{w}',\mathbf{w}}(\mathbf{z})| &\geq |\mathcal{S}\_{\mathbf{w}}(\mathbf{z})||\mathcal{S}\_{\mathbf{w}'}(\mathbf{z})| - |\mathcal{S}\_{\mathbf{w},\mathbf{w}'}(\mathbf{z})||\mathcal{S}\_{\mathbf{w}',\mathbf{w}}(\mathbf{z})| \\ &> |\mathcal{S}\_{\mathbf{w}}(\mathbf{z})||\mathcal{S}\_{\mathbf{w}'}(\mathbf{z})| - |\mathcal{S}\_{\mathbf{w}}(\mathbf{z})||\mathcal{S}\_{\mathbf{w}'}(\mathbf{z})| = 0. \end{split} \tag{41}$$

We complete the proof by setting *K* = max{*<sup>K</sup>*1, *K*2, *K*3, *<sup>K</sup>*3}.

**Lemma 7.** *There exist Kw*,*w* > 0 *and ρ* > 1 *such that pρ* < 1*, and for every word w and w of length k* ≥ *Kw*,*w , the polynomial*

$$\begin{split} D\_{w,w'}(z) &= (1-z)(\mathcal{S}\_{\mathcal{W}}(z)\mathcal{S}\_{w'}(z) - \mathcal{S}\_{w,w'}(z)\mathcal{S}\_{w',w}(z)) \\ &\quad + z^k \left( \mathbf{P}(w)(\mathcal{S}\_{w'}(z) - \mathcal{S}\_{w,w'}(z)) + \mathbf{P}(w')(\mathcal{S}\_{w}(z) - \mathcal{S}\_{w',w}(z)) \right), \end{split} \tag{42}$$

*has exactly one root in the disk* |*z*| ≤ *ρ.*

**Proof.** First note that

$$\begin{split} |\mathcal{S}\_{\mathbf{w}}(z) - \mathcal{S}\_{\mathbf{w}',\mathbf{w}}(z)| &\leq |\mathcal{S}\_{\mathbf{w}}(z)| + |\mathcal{S}\_{\mathbf{w}',\mathbf{w}}(z)| \\ &\leq \frac{1}{1 - p\rho} + \frac{p\rho}{1 - p\rho} = \frac{1 + p\rho}{1 - p\rho}. \end{split} \tag{43}$$

This yields

$$\begin{split} \left| z^k \left( \mathbf{P}(w) (S\_{w'}(z) - S\_{w,w'}(z)) + \mathbf{P}(w') (S\_{w}(z) - S\_{w',w}(z)) \right) \right| \\ \leq (p\rho)^k \left( |S\_w(z) - S\_{w',w}(z)| + |S\_{w'}(z) - S\_{w,w'}(z)| \right) \\ \leq (p\rho)^k \left( \frac{2(1+p\rho)}{1-p\rho} \right) . \end{split} \tag{44}$$

There exist *K*, *K* large enough, such that, for *k* > *K*, we have

> |(*Sw*(*z*)*Sw*(*z*) − *Sw*,*w*(*z*)*Sw*,*<sup>w</sup>*(*z*))| ≥ *β* > 0,

and for *k* > *K*,

$$(p\rho)^k \left(\frac{2(1+p\rho)}{1-p\rho}\right) < (\rho-1)\beta.$$

If we define *Kw*,*w* = max{*K*, *<sup>K</sup>*}, then we have, for *k* ≥ *Kw*,*w* ,

$$\begin{split} (p\rho)^k \left( \frac{2(1+p\rho)}{1-p\rho} \right) &< (\rho - 1)\beta \\ &< |(1-z)(\mathcal{S}\_{\textit{w}}(z)\mathcal{S}\_{\textit{w}'}(z) - \mathcal{S}\_{\textit{w};\textit{w}'}(z)\mathcal{S}\_{\textit{w}',\textit{w}}(z))|. \end{split} \tag{45}$$

by Rouché's theorem, as (1 − *<sup>z</sup>*)(*Sw*(*z*)*Sw*(*z*) − *Sw*,*w*(*z*)*Sw*,*<sup>w</sup>*(*z*)) has only one root in |*z*| ≤ *ρ*, then also *Dw*,*w*(*z*) has exactly one root in |*z*| ≤ *ρ*.

We denote the root within the disk |*z*| ≤ *ρ* of *Dw*,*w*(*z*) by *<sup>α</sup>w*,*w* , and by bootstrapping we obtain

$$\begin{split} a\_{w,w'} &= 1 + \frac{S\_{w'}(1) - S\_{w,w'}(1)}{S\_w(1)S\_{w'}(1) - S\_{w,w'}(1)S\_{w',w}(1)} \mathbf{P}(w) \\ &+ \frac{S\_{w}(1) - S\_{w',w}(1)}{S\_w(1)S\_{w'}(1) - S\_{w,w'}(1)S\_{w',w}(1)} \mathbf{P}(w') + O(p^{2k}).\end{split} \tag{46}$$

We also denote the derivative of *Dw*,*w*(*z*) at the root *<sup>α</sup>w*,*w* , by *β<sup>w</sup>*,*w* , and we obtain

$$\mathcal{S}\_{w,w'} = \mathcal{S}\_{w,w'}(1)\mathcal{S}\_{w',w}(1) - \mathcal{S}\_{w'}(1)\mathcal{S}\_{w'}(1) + O(kp^k). \tag{47}$$

We will refer to these expressions in the residue analysis that we present in the next section.
