*5.3. Analysis*

We next derive an upper bound on the size of the graph *G*. Let *X<sup>n</sup>* = (*X*1, ... , *Xn*) be chosen uniformly at random from the set A ⊆ {−1, 1}*n*, and let <sup>P</sup>*X<sup>n</sup>* be the PMF of *<sup>X</sup>n*. Then,

$$\mathsf{P}\_{\mathcal{X}^n}(\mathsf{x}^n) = \begin{cases} \frac{1}{|\mathcal{A}|} & \text{if } \mathsf{x}^n \in \mathcal{A}\_\prime \\ 0, & \text{if } \mathsf{x}^n \notin \mathcal{A}\_\prime \end{cases} \tag{86}$$

which implies that

$$\mathbb{H}(X^{\mathfrak{n}}) = \log |\mathcal{A}|.\tag{87}$$

The graph *<sup>G</sup>* is an un-directed and simple graph with a vertex set <sup>V</sup>(*G*) = <sup>A</sup> (i.e., the vertices of *G* are in one-to-one correspondence with the binary vectors in the set A). Its set of edges E(*G*) are the edges which connect all pairs of vertices in *G* whose Hamming distance is less than or equal to *<sup>τ</sup>*. For *<sup>d</sup>* <sup>∈</sup> [*τ*], let <sup>E</sup>*d*(*G*) be the set of edges in *<sup>G</sup>* which connect all pairs of vertices in *G* whose Hamming distance is equal to *d*, so

$$|\mathsf{E}(G)| = \sum\_{d=1}^{\tau} |\mathsf{E}\_d(G)|. \tag{88}$$

For *<sup>x</sup><sup>n</sup>* ∈ {−1, 1}*n*, *<sup>d</sup>* <sup>∈</sup> [*n*], and integers *<sup>k</sup>*1, ... , *kd* such that 1 <sup>≤</sup> *<sup>k</sup>*<sup>1</sup> <sup>&</sup>lt; ... <sup>&</sup>lt; *kd* <sup>≤</sup> *<sup>n</sup>*, let

$$\tilde{\mathbf{x}}^{(k\_1, \dots, k\_d)} \triangleq (\mathbf{x}\_1, \dots, \mathbf{x}\_{k\_1 - 1}, \mathbf{x}\_{k\_1 + 1}, \dots, \mathbf{x}\_{k\_d - 1}, \mathbf{x}\_{k\_d + 1}, \dots, \mathbf{x}\_n) \tag{89}$$

be a subvector of *<sup>x</sup><sup>n</sup>* of length *<sup>n</sup>* <sup>−</sup> *<sup>d</sup>*, obtained by dropping the bits of *<sup>x</sup><sup>n</sup>* in positions *<sup>k</sup>*1, ... , *kd*; if *<sup>d</sup>* <sup>=</sup> *<sup>n</sup>*, then (*k*1, ... , *kn*)=(1, ... , *<sup>n</sup>*), and *<sup>x</sup>*\*(*k*1,...,*kd*) is an empty vector. By the chain rule for the Shannon entropy,

$$\begin{split} \mathbf{H}(\mathbf{X}^{n}) - \mathbf{H}(\tilde{\mathbf{X}}^{(k\_{1},...,k\_{d})}) \\ = \mathbf{H}(\mathbf{X}\_{k\_{1}},...,\mathbf{X}\_{k\_{d}} \mid \tilde{\mathbf{X}}^{(k\_{1},...,k\_{d})}) \end{split} \tag{90a}$$

$$\mathbf{x} = -\sum\_{\mathbf{x}^{\mathfrak{n}} \in \{-1, 1\}^{\mathfrak{n}}} \mathbf{P}\_{X^{\mathfrak{n}}}(\mathbf{x}^{\mathfrak{n}}) \log \left( \mathbf{P}\_{X\_{k\_1}, \dots, X\_{k\_d} \mid \bar{X}^{(k\_1, \dots, k\_d)}} \left( \mathbf{x}\_{k\_1}, \dots, \mathbf{x}\_{k\_d} \mid \bar{\mathbf{x}}^{(k\_1, \dots, k\_d)} \right) \right) \tag{90b}$$

$$\hat{\mathbf{x}} = -\frac{1}{|\mathcal{A}|} \sum\_{\mathbf{x}^{\mathbf{u}} \in \mathcal{A}} \log \left( \mathsf{P}\_{\mathbf{X}\_{k\_1}, \dots, \mathbf{X}\_{k\_d} \mid \overline{\mathbf{x}}^{(k\_1, \dots, k\_d)} \left( \mathbf{x}\_{k\_1}, \dots, \mathbf{x}\_{k\_d} \mid \overline{\mathbf{x}}^{(k\_1, \dots, k\_d)} \right) \right), \tag{90c}$$

where equality (90c) holds by (86).

For *<sup>x</sup><sup>n</sup>* ∈ {−1, 1}*n*, *<sup>d</sup>* <sup>∈</sup> [*n*], and integers *<sup>k</sup>*1, ... , *kd* such that 1 <sup>≤</sup> *<sup>k</sup>*<sup>1</sup> <sup>&</sup>lt; ... <sup>&</sup>lt; *kd* <sup>≤</sup> *<sup>n</sup>*, let

$$\underline{\mathbf{x}}^{(k\_1,\dots,k\_d)} \triangleq (\mathbf{x}\_1, \dots, \mathbf{x}\_{k\_1-1}, -\mathbf{x}\_{k\_1}, \mathbf{x}\_{k\_1+1}, \dots, \mathbf{x}\_{k\_d-1}, -\mathbf{x}\_{k\_d}, \mathbf{x}\_{k\_d+1}, \dots, \mathbf{x}\_n),\tag{91}$$

where the bits of *<sup>x</sup><sup>n</sup>* in position *<sup>k</sup>*1, ... , *kd* are flipped (in contrast to *<sup>x</sup>*\*(*k*1,...,*kd*) where the bits of *<sup>x</sup><sup>n</sup>* in these positions are dropped), so *<sup>x</sup>*(*k*1,...,*kd*) ∈ {−1, 1}*<sup>n</sup>* and <sup>d</sup><sup>H</sup> *xn*, *x*(*k*1,...,*kd*) = *d*. Likewise, if *<sup>x</sup>n*, *<sup>y</sup><sup>n</sup>* ∈ {−1, 1}*<sup>n</sup>* satisfy <sup>d</sup>H(*xn*, *<sup>y</sup>n*) <sup>=</sup> *<sup>d</sup>*, then there exist integers *<sup>k</sup>*1, ... , *kd* such that 1 <sup>≤</sup> *<sup>k</sup>*<sup>1</sup> <sup>&</sup>lt; ... <sup>&</sup>lt; *kd* <sup>≤</sup> *<sup>n</sup>* where *<sup>y</sup><sup>n</sup>* <sup>=</sup> *<sup>x</sup>*(*k*1,...,*kd*) (i.e., the integers *<sup>k</sup>*1, ... , *kd* are the positions (in increasing order) where the vectors *x<sup>n</sup>* and *y<sup>n</sup>* differ).

Let us characterize the set A by its cardinality, and the following two natural numbers:

(a) If *<sup>x</sup><sup>n</sup>* ∈ A and *<sup>x</sup>*(*k*1,...,*kd*) ∈ A for any (*k*1, ... , *kd*) such that 1 <sup>≤</sup> *<sup>k</sup>*<sup>1</sup> <sup>&</sup>lt; ... <sup>&</sup>lt; *kd* <sup>≤</sup> *<sup>n</sup>*, then there are at least *md md*(A) vectors *<sup>y</sup>* ∈ A whose subvectors *<sup>y</sup>*\*(*k*1,...,*kd*) coincide with *<sup>x</sup>*\*(*k*1,...,*kd*), i.e., the integer *md* <sup>≥</sup> 2 satisfies

$$m\_d \le \min\_{\substack{\mathbf{x}^n \in \mathcal{A}\_r \\ 1 \le k\_1 < \dots < k\_d \le n}} \left| \left\{ \mathbf{y}^n \in \mathcal{A} : \ \tilde{\mathbf{y}}^{(k\_1, \dots, k\_d)} = \tilde{\mathbf{x}}^{(k\_1, \dots, k\_d)}, \quad \mathbf{\overline{x}}^{(k\_1, \dots, k\_d)} \in \mathcal{A} \right\} \right|. \tag{92}$$

By definition, the integer *md* always exists, and

$$2 \le m\_d \le \min\{2^d, |A|\}. \tag{93}$$

If no information is available about the value of *md*, then it can be taken by default to be equal to 2 (since by assumption the two vectors *<sup>x</sup><sup>n</sup>* ∈ A and *<sup>y</sup><sup>n</sup> <sup>x</sup>*(*k*1,...,*kd*) ∈ A satisfy the equality *<sup>y</sup>*\*(*k*1,...,*kd*) <sup>=</sup> *<sup>x</sup>*\*(*k*1,...,*kd*)).

(b) If *<sup>x</sup><sup>n</sup>* ∈ A and *<sup>x</sup>*(*k*1,...,*kd*) ∈ A for any (*k*1, ... , *kd*) such that 1 <sup>≤</sup> *<sup>k</sup>*<sup>1</sup> <sup>&</sup>lt; ... <sup>&</sup>lt; *kd* <sup>≤</sup> *<sup>n</sup>*, then there are at least *<sup>d</sup> d*(A) vectors *<sup>y</sup><sup>n</sup>* ∈ A whose subvectors *<sup>y</sup>*\*(*k*1,...,*kd*) coincide with *<sup>x</sup>*\*(*k*1,...,*kd*), i.e., the integer *<sup>d</sup>* <sup>≥</sup> 1 satisfies

$$\ell\_d \le \min\_{\substack{\mathbf{x}^n \in \mathcal{A}\_r \\ 1 \le k\_1 < \dots < k\_d \le n}} \left| \left\{ \mathcal{Y}^n \in \mathcal{A} : \ \tilde{\mathcal{Y}}^{(k\_1, \dots, k\_d)} = \tilde{\mathbf{x}}^{(k\_1, \dots, k\_d)}, \quad \tilde{\mathbf{x}}^{(k\_1, \dots, k\_d)} \notin \mathcal{A} \right\} \right|. \tag{94}$$

By definition, the integer *<sup>d</sup>* always exists, and

$$1 \le \ell\_d \le \min\{2^d - 1, \, |A| - 1\}. \tag{95}$$

Likewise, if no information is available about the value of *d*, then it can be taken by default to be equal to 1 (since *<sup>x</sup><sup>n</sup>* ∈ A satisfies the requirement about its subvector *<sup>x</sup>*\*(*k*1,...,*kd*) in (94)).

In general, it would be preferable to have the largest possible values of *md* and *<sup>d</sup>* (i.e., those satisfying inequalities (92) and (94) with equalities, for obtaining a better upper bound on the size of *G* (this point will be clarified in the sequel). If *d* = 1, then *md* = 2 and *<sup>d</sup>* = 1 are the best possible constants (this holds by the definitions in (92) and (94), which can be also verified by the coincidence of the upper and lower bounds in (93) for *d* = 1, as well as those in (95)).

If *<sup>x</sup><sup>n</sup>* ∈ A, then we distinguish between the following two cases:

• If *<sup>x</sup>*(*k*1,...,*kd*) ∈ A, then

$$\left|\mathbb{P}\_{\mathcal{X}\_{k\_1},\ldots,\mathcal{X}\_{k\_d}\mid\bar{X}^{(k\_1,\ldots,k\_d)}}\left(\mathfrak{x}\_{k\_1},\ldots,\mathfrak{x}\_{k\_d}\mid\bar{\mathfrak{x}}^{(k\_1,\ldots,k\_d)}\right)\right| \leq \frac{1}{m\_d!} \tag{96}$$

which holds by the way that *md* is defined in (92), and since *X<sup>n</sup>* is randomly selected to be equiprobable in the set A.

• If *<sup>x</sup>*(*k*1,...,*kd*) ∈ A, then

$$P\_{X\_{k\_1},\ldots,X\_{k\_d}\mid \overline{X}^{(k\_1,\ldots,k\_d)}}\left(\mathfrak{x}\_{k\_1},\ldots,\mathfrak{x}\_{k\_d}\mid \overline{\mathfrak{x}}^{(k\_1,\ldots,k\_d)}\right) \le \frac{1}{\ell\_d},\tag{97}$$

which holds by the way that *<sup>d</sup>* is defined in (94), and since *<sup>X</sup><sup>n</sup>* is equiprobable on <sup>A</sup>. For *d* ∈ [*τ*] and 1 ≤ *k*<sup>1</sup> < ... < *kd* ≤ *n*, it follows from (90), (96) and (97) that

$$\mathbb{H}(X^n) - \mathbb{H}(\tilde{X}^{(k\_1, \dots, k\_d)}) \ge \frac{\log m\_d}{|A|} \sum\_{\mathbf{x}^n} \mathbb{I}\{\mathbf{x}^n \in \mathcal{A}\_\prime \ \mathbb{Z}^{(k\_1, \dots, k\_d)} \in \mathcal{A}\}$$

$$+ \frac{\log \ell\_d}{|A|} \sum\_{\mathbf{x}^n} \mathbb{I}\{\mathbf{x}^n \in \mathcal{A}\_\prime \ \mathbb{Z}^{(k\_1, \dots, k\_d)} \notin \mathcal{A}\},\tag{98}$$

which, by summing on both sides of inequality (98) over all integers *k*1, ... , *kd* such that 1 ≤ *k*<sup>1</sup> < ... < *kd* ≤ *n*, yields

$$\begin{split} \sum\_{\substack{(k\_1,\ldots,k\_d):\\1\le k\_1<\ldots$$

Equality holds in (99) if the minima on the RHS of (92) and (94) are attained by any element in these sets, and if (92) and (94) are satisfied with equalities (i.e., *md* and *<sup>d</sup>* are the maximal integers to satisfy inequalities (92) and (94) for the given set A). Hence, this equality holds in particular for *d* = 1, with the constants *md* = 2 and *<sup>d</sup>* = 1.

The double sum in the first term on the RHS of (99) is equal to

$$\sum\_{\substack{(k\_1,\ldots,k\_d):\\1\le k\_1<\ldots$$

since every pair of adjacent vertices in G that refer to vectors in A whose Hamming distance is equal to *<sup>d</sup>* is of the form *<sup>x</sup><sup>n</sup>* ∈ A and *<sup>x</sup>*(*k*1,...,*kd*) ∈ A, and vice versa, and every edge {*xn*, *<sup>x</sup>*(*k*1,...,*kd*) } ∈ <sup>E</sup>*d*(*G*) is counted twice in the double summation on the LHS of (100). For calculating the double sum in the second term on the RHS of (99), we first calculate the sum of these two double summations:

$$\sum\_{\substack{(k\_1,\ldots,k\_d):\\1\le k\_1<\ldots
$$=\sum\_{\substack{\mathbf{x}^n\in\mathcal{A},\,\mathbf{\overline{x}}^{(k\_1,\ldots,k\_d)}}}\sum\_{\mathbf{x}^n\in\mathcal{A}}\left\{\mathbbm{1}\{\mathbf{x}^n\in\mathcal{A},\,\mathbf{\overline{x}}^{(k\_1,\ldots,k\_d)}\in\mathcal{A}\}+\mathbbm{1}\{\mathbf{x}^n\in\mathcal{A},\,\mathbf{\overline{x}}^{(k\_1,\ldots,k\_d)}\notin\mathcal{A}\}\right\}\tag{101a}$$
$$

$$\sum\_{\substack{(k\_1,\dots,k\_d):\\1\le k\_1<\dots$$

$$=\sum\_{\substack{(k\_1,\ldots,k\_d):\ \\ 1$$

$$\mathcal{A} = \sum\_{(k\_1, \ldots, k\_d) \colon \mathcal{A}} |\mathcal{A}| \tag{101c}$$

$$\begin{aligned} \mathbf{1} & \le k\_1 < \dots < k\_d \le n\\ \mathbf{1} &= \binom{n}{d} |\mathcal{A}| \end{aligned} \tag{101d}$$

so, subtracting (100) from (101d) gives that

$$\sum\_{\substack{(k\_1,\ldots,k\_d):\\1\le k\_1<\ldots$$

Substituting (100) and (102) into the RHS of (99) gives that, for all *d* ∈ [*τ*],

$$\begin{aligned} \sum\_{\substack{(k\_1,\ldots,k\_d):\\1\le k\_1<\ldots$$

$$= \binom{n}{d} \log \ell\_d + \frac{2|\operatorname{E}\_d(G)|}{|\mathcal{A}|} \log \frac{m\_d}{\ell\_d},\tag{103b}$$

with the same necessary and sufficient condition for equality in (103a) as in (99). (Recall that it is in particular an equality for *d* = 1, where in this case *m*<sup>1</sup> = 2 and <sup>1</sup> = 1.)

By the generalized Han's inequality in (28),

$$\sum\_{\substack{(k\_1,\ldots,k\_d):\\1\le k\_1<\ldots$$

$$= \binom{n-1}{d-1} \log |\mathcal{A}| \,\, \tag{104b}$$

where equality (104b) holds by (87). Combining (103) and (104) yields

$$
\binom{n-1}{d-1} \log |\mathcal{A}| \ge \binom{n}{d} \log \ell\_d + \frac{2|\mathsf{E}\_d(G)|}{|\mathcal{A}|} \log \frac{m\_d}{\ell\_d},\tag{105}
$$

and, by the identity ( *n <sup>d</sup>*) <sup>=</sup> *<sup>n</sup> d* ( *n*−1 *<sup>d</sup>*−1), we get

$$|\mathsf{E}\_d(G)| \le \frac{\binom{n-1}{d-1} |\mathcal{A}| \left(\log |\mathcal{A}| - \frac{n}{d} \log \ell\_d\right)}{2 \log \frac{m\_d}{\ell\_d}}.\tag{106}$$

This upper bound is specialized, for *d* = 1, to Theorem 4.2 of [6] (where, by definition, *m*<sup>1</sup> = 2 and <sup>1</sup> = 1). This gives that the number of edges in *G*, connecting pairs of vertices which refer to binary vectors in A whose Hamming distance is 1 from each other, satisfies

$$|\mathsf{E}\_1(G)| \le \frac{1}{2} |\mathcal{A}| \, \log\_2 |\mathcal{A}|. \tag{107}$$

It is possible to select, by default, the values of the integers *md* and *<sup>d</sup>* to be equal to 2 and 1, respectively, independently of the value of *d* ∈ [*τ*]. It therefore follows that the upper bound in (106) can be loosened to

$$|\mathsf{E}\_d(G)| \le \frac{1}{2} \binom{n-1}{d-1} |\mathcal{A}| \, \log\_2 |\mathcal{A}|.\tag{108}$$

This shows that the bound in (108) generalizes the result in Theorem 4.2 of [6], based only on the knowledge of the cardinality of A. Furthermore, the bound (108) can be tightened by the refined bound (106) if the characterization of the set A allows one to assert values for *md* and *<sup>d</sup>* that are larger than the trivial values of 2 and 1, respectively.

In light of (88) and (108), the number of edges in the graph *G* satisfies

$$|\mathbb{E}(G)| \le \frac{1}{2} \sum\_{d=1}^{\tau} \binom{n-1}{d-1} |\mathcal{A}| \log\_2 |\mathcal{A}|.\tag{109}$$

and if *<sup>τ</sup>* <sup>≤</sup> *<sup>n</sup>*+<sup>1</sup> <sup>2</sup> , then it follows that

$$|\mathsf{E}(G)| \le \frac{1}{2} \exp\left( (n-1)\mathsf{H}\_{\mathsf{b}}\left(\frac{\pi - 1}{n - 1}\right) \right) |\mathcal{A}| \log\_2 |\mathcal{A}|.\tag{110}$$

Indeed, the transition from (109) to (110) holds by the inequality

$$\sum\_{k=0}^{n\theta} \binom{n}{k} \le \exp\left(n\mathbb{H}\_{\mathbf{b}}(\theta)\right), \quad \theta \in \left[0, \frac{1}{2}\right],\tag{111}$$

where the latter bound is asymptotically tight in the exponent of *n* (for sufficiently large values of *n*).
