**Proof.**

*i*. We define

$$X\_{n,k}^{(w)} = \begin{cases} 1 & \text{if } w \text{ appears at least once in string } X \\ 0 & \text{otherwise.} \end{cases}$$

This yields

$$\begin{aligned} \mathbb{E}[X\_{n,k}^{(w)}] &= \mathbb{P}(X\_{n,k}^{(w)} = 1) \\ &= 1 - P(X\_{n,k}^{(w)} = 0) \\ &= 1 - [z^n x^0] F\_w(z, x) . \end{aligned} \tag{15}$$

We observe that [*znx*<sup>0</sup>]*Fw*(*<sup>z</sup>*, *<sup>x</sup>*)=[*z<sup>n</sup>*]*Fw*(*<sup>z</sup>*, <sup>0</sup>). By defining *fw*(*z*) = *Fw*(*<sup>z</sup>*, 0) and from (10), we obtain

$$f\_w(z) = \frac{S\_w(z)}{\mathbf{P}(w)z^k + (1-z)S\_w(z)}.\tag{16}$$

Having the above function, we derive the following result.

$$\begin{split} H(z) &= \sum\_{n\geq 0} \mathbb{E}[X\_{n\boldsymbol{k}}] z^n \\ &= \sum\_{n\geq 0} \sum\_{w\in \mathcal{A}^k} (1 - [z^n] f\_w(z)) z^n \\ &= \sum\_{w\in \mathcal{A}^k} \left( \frac{1}{1 - z} - f\_w(z) \right) \\ &= \sum\_{w\in \mathcal{A}^k} \left( \frac{1}{1 - z} - \frac{S\_w(z)}{D\_w(z)} \right). \end{split} \tag{17}$$

*ii*. For this part, we first note that

$$\begin{split} \mathbb{E}\left[\{X\_{n,k}\}\_2\right] &= \mathbb{E}\left[X\_{n,k}^{2}\right] - \mathbb{E}\left[X\_{n,k}\right] \\ &= \mathbb{E}\left[(X\_{n,k}^{(w)} + \dots + X\_{n,k}^{(w^{(r)})})^2\right] - \mathbb{E}\left[X\_{n,k}^{(w)} + \dots + X\_{n,k}^{(w^{(r)})}\right] \\ &= \sum\_{w \in \mathcal{A}^{k}} \mathbb{E}\left[(X\_{n,k}^{(w)})^2\right] + \sum\_{\substack{w, w' \in \mathcal{A}^{k} \\ w \neq w'}} \mathbb{E}\left[X\_{n,k}^{(w)} X\_{n,k}^{(w')}\right] - \sum\_{w \in \mathcal{A}^{k}} \mathbb{E}\left[X\_{n,k}^{(w)}\right] \\ &= \sum\_{\substack{w, w' \in \mathcal{A}^{k} \\ w \neq w'}} \mathbb{E}\left[X\_{n,k}^{(w)} X\_{n,k}^{(w')}\right]. \end{split} \tag{18}$$

Due to properties of indicator random variables, we observe that the expected value of the second factorial moment has only one term:

$$\mathbb{E}\left[\left(X\_{n,k}\right)\_{2}\right] = \sum\_{\substack{w,w' \in \mathcal{A}^{k} \\ w \neq w'}} \mathbb{E}\left[X\_{n,k}^{(w)} X\_{n,k}^{(w')}\right]. \tag{19}$$

We proceed by defining a second indicator variable as following.

$$X\_{n,k}^{(w)}X\_{n,k}^{(w')} = \begin{cases} 1 & \text{if } X\_{n,k}^{(w)} = X\_{n,k}^{(w')} = 1\\ 0 & \text{otherwise,} \end{cases}$$

This gives

$$\begin{split} \mathbb{E}\left[X\_{n,k}^{(w)}X\_{n,k}^{(w')}\right] &= \mathbb{P}\left(X\_{n,k}^{(w)} = 1, X\_{n,k}^{(w')} = 1\right) \\ &= 1 - \mathbb{P}\left(X\_{n,k}^{(w)} = 0 \cup X\_{n,k}^{(w')} = 0\right) \\ &= 1 - \mathbb{P}\left(X\_{n,k}^{(w)} = 0\right) - \mathbb{P}\left(X\_{n,k}^{(w')} = 0\right) + \mathbb{P}\left(X\_{n,k}^{(w)} = 0, X\_{n,k}^{(w')} = 0\right). \end{split}$$

Finally, we are able to express **<sup>E</sup>**[(*Xn*,*<sup>k</sup>*)2] in the following

$$\mathbb{E}[(X\_{\mathfrak{n},k})\_2] = \sum\_{\substack{w,w' \in \mathcal{A}^k \\ w \neq w'}} \left(1 - [z^{\mathfrak{n}}] \, f\_w(z) - [z^{\mathfrak{n}}] f\_{w'}(z) + [z^{\mathfrak{n}}] f\_{ww'}(z)\right), \tag{20}$$

where *fw*,*w*(*z*) = *Fw*,*w*(*<sup>z</sup>*, 0, 0) and [*z<sup>n</sup>*]*Fw*,*w*(*<sup>z</sup>*, 0, <sup>0</sup>)=[*znx*01*x*02]*Fw*,*w*(*<sup>z</sup>*, *x*1, *<sup>x</sup>*2). By (11) we have

$$f\_{w,w'}(z) = \frac{S\_w(z)S\_{w'}(z) - S\_{w,w'}(z)S\_{w',w}(z)}{D\_{w,w'}(z)}\tag{21}$$

Having the above expression, we finally obtain

$$\begin{split} \mathcal{G}\_{k}(z) &= \sum\_{n\geq 0} \mathbb{E}[(X\_{n,k})z] z^{n} \\ &= \sum\_{\substack{w,w' \in A^{k} \ \mathbb{Z}^{d} \\ w \neq w'}} \sum\_{n\geq 0} \left( 1 - [z^{n}] f\_{w}(z) - [z^{n}] f\_{w'}(z) + [z^{n}] f\_{w,w'}(z) \right) z^{n} \\ &= \sum\_{\substack{w,w' \in A^{k} \\ w \neq w'}} \left( \frac{1}{1-z} - f\_{w}(z) - f\_{w'}(z) + f\_{w,w'}(z) \right) \\ &= \sum\_{\substack{w,w' \in A^{k} \\ w \neq w'}} \left( \frac{1}{1-z} - \frac{S\_{w}(z)}{D\_{w}(z)} - \frac{S\_{w'}(z)}{D\_{w'}(z)} + \frac{S\_{w}(z) S\_{w'}(z) - S\_{w,w'}(z) S\_{w',w}(z)}{D\_{w,w'}(z)} \right). \end{split} \tag{22}$$

In the following lemma, we present the generating functions for the first two factorial moments for the *k*th Prefix Complexity in the independent model.

**Lemma 4.** *For H* ˆ *k*(*z*) = ∑*n*≥<sup>0</sup> **E**[*X*<sup>ˆ</sup> *<sup>n</sup>*,*<sup>k</sup>*]*z<sup>n</sup> and G*ˆ *k*(*z*) = ∑*n*≥<sup>0</sup> **E**[(*X*<sup>ˆ</sup> *<sup>n</sup>*,*<sup>k</sup>*)2]*zn, which are the generating functions for* **E**[*X*<sup>ˆ</sup> *<sup>n</sup>*,*<sup>k</sup>*] *and* **E**[(*X*<sup>ˆ</sup> *<sup>n</sup>*,*<sup>k</sup>*)2] *respectively, we have*

*i.*

$$\hat{H}\_k(z) = \sum\_{w \in \mathcal{A}^k} \left( \frac{1}{1 - z} - \frac{1}{1 - (1 - \mathbb{P}(w))z} \right). \tag{23}$$

*ii.*

$$\mathbf{C}\_{k}(z) = \sum\_{\substack{w, w' \in \mathcal{A}^k \\ w \neq w'}} \left( \frac{1}{1 - z} - \frac{1}{1 - (1 - \mathbf{P}(w))z} - \frac{1}{1 - (1 - \mathbf{P}(w'))z} \right)$$

$$+ \sum\_{\substack{w, w' \in \mathcal{A}^k \\ w \neq w'}} \frac{1}{1 - (1 - \mathbf{P}(w) - \mathbf{P}(w'))z}. \tag{24}$$
