Guessing with Distributed Encoders

Abstract

Two correlated sources emit a pair of sequences, each of which is observed by a different encoder. Each encoder produces a rate-limited description of the sequence it observes, and the two descriptions are presented to a guessing device that repeatedly produces sequence pairs until correct. The number of guesses until correct is random, and it is required that it have a moment (of some prespecified order) that tends to one as the length of the sequences tends to infinity. The description rate pairs that allow this are characterized in terms of the Rényi entropy and the Arimoto–Rényi conditional entropy of the joint law of the sources. This solves the guessing analog of the Slepian–Wolf distributed source-coding problem. The achievability is based on random binning, which is analyzed using a technique by Rosenthal.

1. Introduction

In the Massey–Arıkan guessing problem [1,2], a random variable X is drawn from a finite set $\mathcal{X}$ according to some probability mass function (PMF) $P_X$, and it has to be determined by making guesses of the form “Is X equal to x?” until the guess is correct. The guessing order is determined by a guessing function G, which is a bijective function from $\mathcal{X}$ to $\{1,\dots ,|\mathcal{X}\left|\right\}$. Guessing according to G proceeds as follows: the first guess is the element ${\widehat{x}}_1\in \mathcal{X}$ satisfying $G\left({\widehat{x}}_1\right)=1$; the second guess is the element ${\widehat{x}}_2\in \mathcal{X}$ satisfying $G\left({\widehat{x}}_2\right)=2$, and so on. Consequently, $G\left(X\right)$ is the number of guesses needed to guess X. Arıkan [2] showed that for any $\rho >0$, the $\rho$th moment of the number of guesses required by an optimal guesser $G^{\ast }$ to guess X is bounded by:

$$\begin{array}{c}\hfill \frac{1}{{(1+ln\left|\mathcal{X}\right|)}^{\rho }}{2}^{\rho H_{1/(1+\rho )}\left(X\right)}\le \mathrm{E}\left[G^{\ast }{\left(X\right)}^{\rho }\right]\le 2^{\rho H_{1/(1+\rho )}\left(X\right)},\end{array}$$

(1)

where $ln(·)$ denotes the natural logarithm, and $H_{1/(1+\rho )}\left(X\right)$ denotes the Rényi entropy of order $\frac{1}{1+\rho }$, which is defined in Section 3 ahead (refinements of (1) were recently derived in [3]).

Guessing with an encoder is depicted in Figure 1. Here, prior to guessing X, the guesser is provided some side information about X in the form of $f\left(X\right)$, where $f:\mathcal{X}\to \{1,\dots ,\mathsf{M}\}$ is a function taking on at most $\mathsf{M}$ different values (“labels”). Accordingly, a guessing function $G(·|·)$ is a function from $\mathcal{X}\times \{1,\dots ,\mathsf{M}\}$ to $\{1,\dots ,|\mathcal{X}\left|\right\}$ such that for every label $m\in \{1,\dots ,\mathsf{M}\}$, $G(·|m):\mathcal{X}\to \{1,\dots ,\left|\mathcal{X}\right|\}$ is bijective. If, among all encoders, $f^{\ast }$ minimizes the $\rho$th moment of the number of guesses required by an optimal guesser to guess X after observing $f\left(X\right)$, then [4] (Corollary 7):

$$\begin{array}{c}\hfill \frac{1}{{(1+ln\left|\mathcal{X}\right|)}^{\rho }}{2}^{\rho [H_{1/(1+\rho )}\left(X\right)-log\mathsf{M}]}\le \mathrm{E}\left[G^{\ast }{\left(X\right|f^{\ast }\left(X\right))}^{\rho }\right]\le 1+2^{\rho [H_{1/(1+\rho )}\left(X\right)-log\mathsf{M}+1]}.\end{array}$$

(2)

Thus, in guessing a sequence of independent and identically distributed (IID) random variables, a description rate of approximately $H_{1/(1+\rho )}\left(X\right)$ bits per symbol is needed to drive the $\rho$th moment of the number of guesses to one as the sequence length tends to infinity [4,5] (see Section 2 for more related work).

In this paper, we generalize the single-encoder setting from Figure 1 to the setting with distributed encoders depicted in Figure 2, which is the analog of Slepian–Wolf coding [6] for guessing: A source generates a sequence of pairs ${\left\{(X_i,Y_i)\right\}}_{i=1}^n$ over a finite alphabet $\mathcal{X}\times \mathcal{Y}$. The sequence $X^n$ is described by one of $\lfloor 2^{n{\mathsf{R}}_{\mathsf{X}}}\rfloor$ labels and the sequence $Y^n$ by one of $\lfloor 2^{n{\mathsf{R}}_{\mathsf{Y}}}\rfloor$ labels using functions:

$$\begin{array}{cc}\hfill f_n:{\mathcal{X}}^n& \to \{1,\dots ,\lfloor 2^{n{\mathsf{R}}_{\mathsf{X}}}\rfloor \},\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill g_n:{\mathcal{Y}}^n& \to \{1,\dots ,\lfloor 2^{n{\mathsf{R}}_{\mathsf{Y}}}\rfloor \},\hfill \end{array}$$

(4)

where ${\mathsf{R}}_{\mathsf{X}}\ge 0$ and ${\mathsf{R}}_{\mathsf{Y}}\ge 0$. Based on $f_n\left(X^n\right)$ and $g_n\left(Y^n\right)$, a guesser repeatedly produces guesses of the form $({\widehat{x}}^n,{\widehat{y}}^n)$ until $({\widehat{x}}^n,{\widehat{y}}^n)=(X^n,Y^n)$.

For a fixed $\rho >0$, a rate pair $({\mathsf{R}}_{\mathsf{X}},{\mathsf{R}}_{\mathsf{Y}})\in {\mathbb{R}}_{\ge 0}^2$ is called achievable if there exists a sequence of encoders and guessing functions ${\left\{(f_n,g_n,G_n)\right\}}_{n=1}^{\infty }$ such that the $\rho$th moment of the number of guesses tends to one as n tends to infinity, i.e.,

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}\mathrm{E}\left[G_n{\left(X^n,Y^n|f_n\left(X^n\right),g_n\left(Y^n\right)\right)}^{\rho }\right]=1.\end{array}$$

(5)

Our main contribution is Theorem 1, which characterizes the achievable rate pairs. For a fixed $\rho >0$, let the region $\mathcal{R}\left(\rho \right)$ comprise all rate pairs $({\mathsf{R}}_{\mathsf{X}},{\mathsf{R}}_{\mathsf{Y}})\in {\mathbb{R}}_{\ge 0}^2$ satisfying the following inequalities simultaneously:

$$\begin{array}{cc}\hfill {\mathsf{R}}_{\mathsf{X}}& \ge \underset{n\to \infty }{lim\; sup}\frac{H_{\tilde{\rho }}\left(X^n\right|Y^n)}{n},\hfill \end{array}$$

(6)

$$\begin{array}{cc}\hfill {\mathsf{R}}_{\mathsf{Y}}& \ge \underset{n\to \infty }{lim\; sup}\frac{H_{\tilde{\rho }}\left(Y^n\right|X^n)}{n},\hfill \end{array}$$

(7)

$$\begin{array}{cc}\hfill {\mathsf{R}}_{\mathsf{X}}+{\mathsf{R}}_{\mathsf{Y}}& \ge \underset{n\to \infty }{lim\; sup}\frac{H_{\tilde{\rho }}(X^n,Y^n)}{n},\hfill \end{array}$$

(8)

where the Rényi entropy $H_{\alpha }(·)$ and the Arimoto–Rényi conditional entropy $H_{\alpha }(·|·)$ of order $\alpha$ are both defined in Section 3 ahead, and throughout the paper,

$$\begin{array}{c}\hfill \tilde{\rho }\triangleq \frac{1}{1+\rho }.\end{array}$$

(9)

Theorem 1.

For any $\rho >0$, all rate pairs in the interior of $\mathcal{R}\left(\rho \right)$ are achievable, while those outside $\mathcal{R}\left(\rho \right)$ are not. If ${\left\{(X_i,Y_i)\right\}}_{i=1}^{\infty }$ are IID according to $P_{XY}$, then (6)–(8) reduce to:

$$\begin{array}{cc}\hfill {\mathsf{R}}_{\mathsf{X}}& \ge H_{\tilde{\rho }}\left(X\right|Y),\hfill \end{array}$$

(10)

$$\begin{array}{cc}\hfill {\mathsf{R}}_{\mathsf{Y}}& \ge H_{\tilde{\rho }}\left(Y\right|X),\hfill \end{array}$$

(11)

$$\begin{array}{cc}\hfill {\mathsf{R}}_{\mathsf{X}}+{\mathsf{R}}_{\mathsf{Y}}& \ge H_{\tilde{\rho }}(X,Y).\hfill \end{array}$$

(12)

Proof. 

The converse follows from Corollary 1 in Section 4; the achievability follows from Corollary 2 in Section 5; and the reduction of (6)–(8) to (10)–(12) in the IID case follows from (19) and (20) ahead. ☐

The rate region defined by (10)–(12) resembles the rate region of Slepian–Wolf coding [6] (Theorem 15.4.1); the difference is that the Shannon entropy and conditional entropy are replaced by their Rényi counterparts. The rate regions are related as follows:

Remark 1.

For memoryless sources and $\rho >0$, the region $\mathcal{R}\left(\rho \right)$ is contained in the Slepian–Wolf region. Typically, the containment is strict.

Proof. 

The containment follows from the monotonicity of the Arimoto–Rényi conditional entropy: (9) implies that $\tilde{\rho }\in (0,1)$, so, by [7] (Proposition 5), $H_{\tilde{\rho }}\left(X\right|Y)\ge H\left(X\right|Y)$, $H_{\tilde{\rho }}\left(Y\right|X)\ge H\left(Y\right|X)$, and $H_{\tilde{\rho }}(X,Y)\ge H(X,Y)$. As for the strict containment, first note that the Slepian–Wolf region contains at least one rate pair $({\mathsf{R}}_{\mathsf{X}},{\mathsf{R}}_{\mathsf{Y}})$ satisfying ${\mathsf{R}}_{\mathsf{X}}+{\mathsf{R}}_{\mathsf{Y}}=H(X,Y)$. Consequently, if $H_{\tilde{\rho }}(X,Y)>H(X,Y)$, then the containment is strict. Because $H_{\tilde{\rho }}(X,Y)>H(X,Y)$ unless $(X,Y)$ is distributed uniformly over its support [8], the containment is typically strict.

The claim can also be shown operationally: The probability of error is equal to the probability that more than one guess is needed, and for every $\rho >0$,

$$\begin{array}{cc}\hfill Pr\left[G_n\left(X^n,Y^n|f_n\left(X^n\right),g_n\left(Y^n\right)\right)\ge 2\right]& =Pr\left[G_n{\left(X^n,Y^n|f_n\left(X^n\right),g_n\left(Y^n\right)\right)}^{\rho }-1\ge 2^{\rho }-1\right]\hfill \end{array}$$

(13)

$$\begin{array}{c}\le \frac{\mathrm{E}\left[G_n{\left(X^n,Y^n|f_n\left(X^n\right),g_n\left(Y^n\right)\right)}^{\rho }\right]-1}{2^{\rho }-1},\hfill \end{array}$$

(14)

where (14) follows from Markov’s inequality. Thus, the probability of error tends to zero if the $\rho$th moment of the number of guesses tends to one.  ☐

Despite the resemblance between (10)–(12) and the Slepian–Wolf region, there is an important difference: while Slepian–Wolf coding allows separate encoding with the same sum rate as with joint encoding, this is not necessarily true in our setting:

Remark 2.

Although the sum rate constraint (12) is the same as in single-source guessing [5], separate encoding of $X^n$ and $Y^n$ may require a larger sum rate than joint encoding of $X^n$ and $Y^n$.

Proof. 

If $H_{\tilde{\rho }}\left(X\right|Y)+H_{\tilde{\rho }}\left(Y\right|X)>H_{\tilde{\rho }}(X,Y)$, then (10) and (11) together impose a stronger constraint on the sum rate than (12). For example, if:

$$\begin{array}{ccc}{P}_{XY}(x,y)& y=0& y=1\\ x=0& 0.65& 0.17\\ x=1& 0.17& 0.01\end{array}$$

and $\rho =1$, then $H_{1/2}\left(X\right|Y)+H_{1/2}\left(Y\right|X)\approx 1.61$ bits, so separate (distributed) encoding requires a sum rate exceeding $1.61$ bits as opposed to joint encoding, which is possible with $H_{1/2}(X,Y)\approx 1.58$ bits (in Slepian–Wolf coding, this cannot happen because $H(X,Y)-H\left(X\right|Y)-H(Y\left|X\right)=I(X;Y)\ge 0$).  ☐

The guessing problem is related to the task-encoding problem, where based on $f_n\left(X^n\right)$ and $g_n\left(Y^n\right)$, the decoder outputs a list that is guaranteed to contain $(X^n,Y^n)$, and the $\rho$th moment of the list size is required to tend to one as n tends to infinity. While, in the single-source setting, the guessing problem and the task-encoding problem have the same asymptotics [4], this is not the case in the distributed setting:

Remark 3.

For memoryless sources, the task-encoding region from [9] is strictly smaller than the guessing region $\mathcal{R}\left(\rho \right)$ unless X and Y are independent.

Proof. 

In the IID case, the task-encoding region is the set of all rate pairs $({\mathsf{R}}_{\mathsf{X}},{\mathsf{R}}_{\mathsf{Y}})\in {\mathbb{R}}_{\ge 0}^2$ satisfying the following inequalities [9] (Theorem 1):

$$\begin{array}{cc}\hfill {\mathsf{R}}_{\mathsf{X}}& \ge H_{\tilde{\rho }}\left(X\right),\hfill \end{array}$$

(15)

$$\begin{array}{cc}\hfill {\mathsf{R}}_{\mathsf{Y}}& \ge H_{\tilde{\rho }}\left(Y\right),\hfill \end{array}$$

(16)

$$\begin{array}{cc}\hfill {\mathsf{R}}_{\mathsf{X}}+{\mathsf{R}}_{\mathsf{Y}}& \ge H_{\tilde{\rho }}(X,Y)+K_{\tilde{\rho }}(X;Y),\hfill \end{array}$$

(17)

where $K_{\alpha }(X;Y)$ is a Rényi measure of dependence studied in [10] (when $\alpha$ is one, $K_{\alpha }(X;Y)$ is the mutual information). The claim now follows from the following observations: By [7] (Theorem 2), $H_{\tilde{\rho }}\left(X\right)\ge H_{\tilde{\rho }}\left(X\right|Y)$ with equality if and only if X and Y are independent; similarly, $H_{\tilde{\rho }}\left(Y\right)\ge H_{\tilde{\rho }}\left(Y\right|X)$ with equality if and only if X and Y are independent; and by [10] (Theorem 2), $K_{\tilde{\rho }}(X;Y)\ge 0$ with equality if and only if X and Y are independent.  ☐

The rest of this paper is structured as follows: in Section 2, we review other guessing settings; in Section 3, we recall the Rényi information measures and prove some auxiliary lemmas; in Section 4, we prove the converse theorem; and in Section 5, we prove the achievability theorem, which is based on random binning and, in the case $\rho >1$, is analyzed using a technique by Rosenthal [11].

2. Related Work

Tighter versions of (1) can be found in [3,12]. The large deviation behavior of guessing was studied in [13,14]. The relation between guessing and variable-length lossless source coding was explored in [3,15,16].

Mismatched guessing, where the assumed distribution of X does not match its actual distribution, was studied in [17], along with guessing under source uncertainty, where the PMF of X belongs to some known set, and a guesser was sought with good worst-case performance over that set. Guessing subject to distortion, where instead of guessing X, it suffices to guess an $\widehat{X}$ that is close to X according to some distortion measure, was treated in [18].

If the guesser observes some side information Y, then the $\rho$th moment of the number of guesses required by an optimal guesser is bounded by [2]:

$$\begin{array}{c}\hfill \frac{1}{{(1+ln\left|\mathcal{X}\right|)}^{\rho }}{2}^{\rho H_{\tilde{\rho }}\left(X\right|Y)}\le \mathrm{E}\left[G^{\ast }{\left(X\right|Y)}^{\rho }\right]\le 2^{\rho H_{\tilde{\rho }}\left(X\right|Y)},\end{array}$$

(18)

where $H_{\tilde{\rho }}\left(X\right|Y)$ denotes the Arimoto–Rényi conditional entropy of order $\tilde{\rho }=\frac{1}{1+\rho }$, which is defined in Section 3 ahead (refinements of (18) were recently derived in [3]). Guessing is related to the cutoff rate of a discrete memoryless channel, which is the supremum over all rates for which the $\rho$th moment of the number of guesses needed by the decoder to guess the message can be driven to one as the block length tends to infinity. In [2,19], the cutoff rate was expressed in terms of Gallager’s $E_0$ function [20]. Joint source-channel guessing was considered in [21].

Guessing with an encoder, i.e., the situation where the side information can be chosen, was studied in [4], where it was also shown that guessing and task encoding [22] have the same asymptotics. With distributed encoders, however, task encoding [9] and guessing no longer have the same asymptotics; see Remark 3. Lower and upper bounds for guessing with a helper, i.e., an encoder that does not observe X, but has access to a random variable that is correlated with X, can be found in [5].

3. Preliminaries

Throughout the paper, $log(·)$ denotes the base-two logarithm. When clear from the context, we often omit sets and subscripts; for example, we write ${\sum }_x$ for ${\sum }_{x\in \mathcal{X}}$ and $P\left(x\right)$ for $P_X\left(x\right)$. The Rényi entropy [23] of order $\alpha$ is defined for positive $\alpha$ other than one as:

$$\begin{array}{c}\hfill H_{\alpha }\left(X\right)\triangleq \frac{1}{1-\alpha }log\sum _{x}P{\left(x\right)}^{\alpha }.\end{array}$$

(19)

In the limit as $\alpha$ tends to one, the Shannon entropy is recovered, i.e., ${lim}_{\alpha \to 1}{H}_{\alpha }\left(X\right)=H\left(X\right)$. The Arimoto–Rényi conditional entropy [24] of order $\alpha$ is defined for positive $\alpha$ other than one as:

$$\begin{array}{c}\hfill H_{\alpha }\left(X\right|Y)\triangleq \frac{\alpha }{1-\alpha }log\sum _y{\left[\sum _{x}P{(x,y)}^{\alpha }\right]}^{\frac{1}{\alpha }}.\end{array}$$

(20)

In the limit as $\alpha$ tends to one, the Shannon conditional entropy is recovered, i.e., ${lim}_{\alpha \to 1}{H}_{\alpha }\left(X\right|Y)=H\left(X\right|Y)$. The properties of the Arimoto–Rényi conditional entropy were studied in [7,24,25].

In the rest of this section, we recall some properties of the Arimoto–Rényi conditional entropy that will be used in Section 4 (Lemmas 1–3), and we prove auxiliary results for Section 5 (Lemmas 4–7).

Lemma 1 

([7], Theorem 2). Let $\alpha >0$, and let $P_{XYZ}$ be a PMF over the finite set $\mathcal{X}\times \mathcal{Y}\times \mathcal{Z}$. Then,

$$\begin{array}{c}\hfill H_{\alpha }\left(X\right|Y,Z)\le H_{\alpha }\left(X\right|Z)\end{array}$$

(21)

with equality if and only if $X⊸-Z⊸-Y$ form a Markov chain.

Lemma 2 

([7], Proposition 4). Let $\alpha >0$, and let $P_{XYZ}$ be a PMF over the finite set $\mathcal{X}\times \mathcal{Y}\times \mathcal{Z}$. Then,

$$\begin{array}{c}\hfill H_{\alpha }(X,Y|Z)\ge H_{\alpha }\left(X\right|Z)\end{array}$$

(22)

with equality if and only if Y is uniquely determined by X and Z.

Lemma 3 

([7], Theorem 3). Let $\alpha >0$, and let $P_{XYZ}$ be a PMF over the finite set $\mathcal{X}\times \mathcal{Y}\times \mathcal{Z}$. Then,

$$\begin{array}{c}\hfill H_{\alpha }\left(X\right|Y,Z)\ge H_{\alpha }\left(X\right|Z)-log\left|\mathcal{Y}\right|.\end{array}$$

(23)

Lemma 4 

([20], Problem 4.15(f)). Let $\mathcal{Y}$ be a finite set, and let $f:\mathcal{Y}\to {\mathbb{R}}_{\ge 0}$. Then, for all $p\in (0,1]$,

$$\begin{array}{c}\hfill {\left[\sum _{y}f\left(y\right)\right]}^p\le \sum _{y}f{\left(y\right)}^p.\end{array}$$

(24)

Proof. 

If ${\sum }_{y}f\left(y\right)=0$, then (24) holds because the left-hand side (LHS) and the right-hand side (RHS) are both zero. If ${\sum }_{y}f\left(y\right)>0$, then:

$$\begin{array}{cc}\hfill \sum _{y}f{\left(y\right)}^p& ={\left[\sum _{y^{\prime }}f\left(y^{\prime }\right)\right]}^p\sum _y{\left[\frac{f\left(y\right)}{{\sum }_{y^{\prime }}f\left(y^{\prime }\right)}\right]}^p\hfill \end{array}$$

(25)

$$\begin{array}{c}\ge {\left[\sum _{y^{\prime }}f\left(y^{\prime }\right)\right]}^p\sum _y\frac{f\left(y\right)}{{\sum }_{y^{\prime }}f\left(y^{\prime }\right)}\hfill \end{array}$$

(26)

$$\begin{array}{c}={\left[\sum _{y^{\prime }}f\left(y^{\prime }\right)\right]}^p,\hfill \end{array}$$

(27)

where (26) holds because $p\in (0,1]$ and $f\left(y\right)/{\sum }_{y^{\prime }}f\left(y^{\prime }\right)\in [0,1]$ for every $y\in \mathcal{Y}$.  ☐

Lemma 5.

Let a, b, and c be nonnegative integers. Then, for all $p>0$,

$$\begin{array}{c}\hfill {(1+a+b+c)}^p\le 1+4^p(a^p+b^p+c^p)\end{array}$$

(28)

(the restriction to integers cannot be omitted; for example, (28) does not hold if $a=b=c=0.1$ and $p=2$).

Proof. 

If $p\in (0,1]$, then (28) follows from Lemma 4 because $4^p\ge 1$. If $p>1$, then the cases with $a+b+c\in \{0,1,2\}$ can be checked individually. For $a+b+c\ge 3$,

$$\begin{array}{cc}\hfill {(1+a+b+c)}^p& ={\left[\frac{3}{a+b+c}+3\right]}^p·{\left[\frac{a+b+c}{3}\right]}^p\hfill \end{array}$$

(29)

$$\begin{array}{c}\le 4^p·{\left[\frac{a+b+c}{3}\right]}^p\hfill \end{array}$$

(30)

$$\begin{array}{cc}\hfill & \le 4^p·\frac{a^p+b^p+c^p}{3}\hfill \end{array}$$

(31)

$$\begin{array}{c}\le 1+4^p(a^p+b^p+c^p),\hfill \end{array}$$

(32)

where (30) holds because $a+b+c\ge 3$, and (31) follows from Jensen’s inequality because $z\mapsto z^p$ is convex on ${\mathbb{R}}_{\ge 0}$ since $p>1$.  ☐

Lemma 6.

Let a, b, c, and d be nonnegative real numbers. Then, for all $p>0$,

$$\begin{array}{c}\hfill {(a+b+c+d)}^p\le 4^p(a^p+b^p+c^p+d^p).\end{array}$$

(33)

Proof. 

If $p\in (0,1]$, then (33) follows from Lemma 4 because $4^p\ge 1$. If $p>1$, then:

$$\begin{array}{cc}\hfill {(a+b+c+d)}^p& =4^p·{\left[\frac{a+b+c+d}{4}\right]}^p\hfill \end{array}$$

(34)

$$\begin{array}{c}\le 4^p·\frac{a^p+b^p+c^p+d^p}{4}\hfill \end{array}$$

(35)

$$\begin{array}{c}\le 4^p(a^p+b^p+c^p+d^p),\hfill \end{array}$$

(36)

where (35) follows from Jensen’s inequality because $z\mapsto z^p$ is convex on ${\mathbb{R}}_{\ge 0}$ since $p>1$. ☐

Lemma 7 

(Rosenthal). Let $p>1$, and let $X_1,\dots ,X_n$ be independent random variables that are either zero or one. Then, $X\triangleq {\sum }_{i=1}^n{X}_i$ satisfies:

$$\begin{array}{c}\hfill \mathrm{E}\left[X^p\right]\le 2^{p^2}max\{\mathrm{E}\left[X\right],\mathrm{E}{\left[X\right]}^p\}.\end{array}$$

(37)

Proof. 

This is a special case of [11] (Lemma 1). For convenience, we also provide a self-contained proof:

$$\begin{array}{cc}\hfill \mathrm{E}\left[X^p\right]& =\mathrm{E}\left[\sum _{i\in \{1,\dots ,n\}}{X}_i·{\left\{\sum _{j\in \{1,\dots ,n\}}{X}_j\right\}}^{p-1}\right]\hfill \end{array}$$

(38)

$$\begin{array}{cc}\hfill & =\mathrm{E}\left[\sum _{i\in \{1,\dots ,n\}}{X}_i·{\left\{1+\sum _{j\in \{1,\dots ,n\}\setminus \left\{i\right\}}{X}_j\right\}}^{p-1}\right]\hfill \end{array}$$

(39)

$$\begin{array}{cc}\hfill & =\sum _{i\in \{1,\dots ,n\}}\mathrm{E}\left[X_i·{\left\{1+\sum _{j\in \{1,\dots ,n\}\setminus \left\{i\right\}}{X}_j\right\}}^{p-1}\right]\hfill \end{array}$$

(40)

$$\begin{array}{cc}\hfill & =\sum _{i\in \{1,\dots ,n\}}\mathrm{E}\left[X_i\right]·\mathrm{E}\left[{\left\{1+\sum _{j\in \{1,\dots ,n\}\setminus \left\{i\right\}}{X}_j\right\}}^{p-1}\right]\hfill \end{array}$$

(41)

$$\begin{array}{cc}\hfill & \le \sum _{i\in \{1,\dots ,n\}}\mathrm{E}\left[X_i\right]·\mathrm{E}\left[{\left\{1+\sum _{j\in \{1,\dots ,n\}}{X}_j\right\}}^{p-1}\right]\hfill \end{array}$$

(42)

$$\begin{array}{cc}\hfill & =\mathrm{E}\left[X\right]·\mathrm{E}\left[{(1+X)}^{p-1}\right]\hfill \end{array}$$

(43)

$$\begin{array}{cc}\hfill & \le \mathrm{E}\left[X\right]·2^{p-1}·(1+\mathrm{E}\left[X^{p-1}\right])\hfill \end{array}$$

(44)

$$\begin{array}{cc}\hfill & =2^{p-1}(\mathrm{E}\left[X\right]+\mathrm{E}\left[X\right]\mathrm{E}\left[X^{p-1}\right])\hfill \end{array}$$

(45)

$$\begin{array}{cc}\hfill & \le 2^{p-1}\left(\mathrm{E}\left[X\right]+\mathrm{E}\left[X\right]\mathrm{E}{\left[X^p\right]}^{\frac{p-1}{p}}\right)\hfill \end{array}$$

(46)

$$\begin{array}{cc}\hfill & \le 2^{p}max\left\{\mathrm{E}\left[X\right],\mathrm{E}\left[X\right]\mathrm{E}{\left[X^p\right]}^{\frac{p-1}{p}}\right\},\hfill \end{array}$$

(47)

where (39) holds because each $X_i$ is either zero or one; (41) holds because $X_1,\dots ,X_n$ are independent; (42) holds because $z\mapsto z^{p-1}$ is increasing on ${\mathbb{R}}_{\ge 0}$ for $p>1$; (44) holds because for real numbers $a\ge 0$, $b\ge 0$, and $r>0$, we have ${(a+b)}^r\le {(2max\{a,b\})}^r=2^{r}max\{a^r,b^r\}\le 2^r(a^r+b^r)$; and (46) follows from Jensen’s inequality because $z\mapsto z^{(p-1)/p}$ is concave on ${\mathbb{R}}_{\ge 0}$ for $p>1$.

We now consider two cases depending on which term on the RHS of (47) achieves the maximum: If the maximum is achieved by $\mathrm{E}\left[X\right]$, then $\mathrm{E}\left[X^p\right]\le 2^p\mathrm{E}\left[X\right]$, which implies (37) because $2^p\le 2^{p^2}$ since $p>1$. If the maximum is achieved by $\mathrm{E}\left[X\right]\mathrm{E}{\left[X^p\right]}^{(p-1)/p}$, then:

$$\begin{array}{c}\hfill \mathrm{E}\left[X^p\right]\le 2^p\mathrm{E}\left[X\right]\mathrm{E}{\left[X^p\right]}^{\frac{p-1}{p}}.\end{array}$$

(48)

Rearranging (48), we obtain:

$$\begin{array}{c}\hfill \mathrm{E}\left[X^p\right]\le 2^{p^2}\mathrm{E}{\left[X\right]}^p,\end{array}$$

(49)

so (37) holds also in this case. ☐

4. Converse

In this section, we prove a nonasymptotic and an asymptotic converse result (Theorem 2 and Corollary 1, respectively).

Theorem 2.

Let $U⊸-X⊸-Y⊸-V$ form a Markov chain over the finite set $\mathcal{U}\times \mathcal{X}\times \mathcal{Y}\times \mathcal{V}$, and let $\tau \triangleq 1+ln|\mathcal{X}\times \mathcal{Y}|$. Then, for every $\rho >0$ and for every guesser, the ρth moment of the number of guesses it takes to guess the pair $(X,Y)$ based on the side information $(U,V)$ satisfies:

$$\begin{array}{cc}\mathrm{E}\left[G{(X,Y|U,V)}^{\rho }\right]\ge max\{\hfill & 2^{\rho (H_{\tilde{\rho }}\left(X\right|Y)-log\left|\mathcal{U}\right|-log\tau )},\hfill \\ & 2^{\rho (H_{\tilde{\rho }}\left(Y\right|X)-log\left|\mathcal{V}\right|-log\tau )},\hfill \\ & 2^{\rho (H_{\tilde{\rho }}(X,Y)-log|\mathcal{U}\times \mathcal{V}|-log\tau )}\}.\hfill \end{array}$$

(50)

Proof. 

We view (50) as three lower bounds corresponding to the three terms in the maximization on its RHS. The lower bound involving $H_{\tilde{\rho }}(X,Y)$ holds because:

$$\begin{array}{cc}\hfill \mathrm{E}\left[G{(X,Y|U,V)}^{\rho }\right]& \ge 2^{\rho (H_{\tilde{\rho }}(X,Y|U,V)-log\tau )}\hfill \end{array}$$

(51)

$$\begin{array}{cc}\hfill & \ge 2^{\rho (H_{\tilde{\rho }}(X,Y)-log|\mathcal{U}\times \mathcal{V}|-log\tau )},\hfill \end{array}$$

(52)

where (51) follows from (18) and (52) follows from Lemma 3. The lower bound involving $H_{\tilde{\rho }}\left(X\right|Y)$ holds because:

$$\begin{array}{cc}\hfill \mathrm{E}\left[G{(X,Y|U,V)}^{\rho }\right]& \ge 2^{\rho (H_{\tilde{\rho }}(X,Y|U,V)-log\tau )}\hfill \end{array}$$

(53)

$$\begin{array}{cc}\hfill & \ge 2^{\rho (H_{\tilde{\rho }}(X,Y|U,V,Y)-log\tau )}\hfill \end{array}$$

(54)

$$\begin{array}{cc}\hfill & =2^{\rho (H_{\tilde{\rho }}\left(X\right|U,V,Y)-log\tau )}\hfill \end{array}$$

(55)

$$\begin{array}{cc}\hfill & =2^{\rho (H_{\tilde{\rho }}\left(X\right|U,Y)-log\tau )}\hfill \end{array}$$

(56)

$$\begin{array}{cc}\hfill & \ge 2^{\rho (H_{\tilde{\rho }}\left(X\right|Y)-log\left|\mathcal{U}\right|-log\tau )},\hfill \end{array}$$

(57)

where (53) follows from (18); (54) follows from Lemma 1; (55) follows from Lemma 2; (56) follows from Lemma 1 because $X⊸-(U,Y)⊸-V$ form a Markov chain; and (57) follows from Lemma 3. The lower bound involving $H_{\tilde{\rho }}\left(Y\right|X)$ is analogous to the one with $H_{\tilde{\rho }}\left(X\right|Y)$. ☐

Corollary 1.

For any $\rho >0$, rate pairs outside $\mathcal{R}\left(\rho \right)$ are not achievable.

Proof. 

We first show that (8) is necessary for a rate pair $({\mathsf{R}}_{\mathsf{X}},{\mathsf{R}}_{\mathsf{Y}})\in {\mathbb{R}}_{\ge 0}^2$ to be achievable. Indeed, if (8) does not hold, then there exists an $ϵ>0$ such that for infinitely many n,

$$\begin{array}{c}\hfill \frac{H_{\tilde{\rho }}(X^n,Y^n)}{n}\ge {\mathsf{R}}_{\mathsf{X}}+{\mathsf{R}}_{\mathsf{Y}}+ϵ.\end{array}$$

(58)

Using Theorem 2 with ${\mathcal{X}}^{\prime }\triangleq {\mathcal{X}}^n$, ${\mathcal{Y}}^{\prime }\triangleq {\mathcal{Y}}^n$, $\mathcal{U}\triangleq \{1,\dots ,\lfloor 2^{n{\mathsf{R}}_{\mathsf{X}}}\rfloor \}$, $\mathcal{V}\triangleq \{1,\dots ,\lfloor 2^{n{\mathsf{R}}_{\mathsf{Y}}}\rfloor \}$, $P_{X^{\prime }{Y}^{\prime }}\triangleq P_{X^n{Y}^n}$, $U\triangleq f_n\left(X^n\right)$, $V\triangleq g_n\left(Y^n\right)$, and ${\tau }_n=1+nln|\mathcal{X}\times \mathcal{Y}|$ leads to:

$$\begin{array}{cc}\hfill \mathrm{E}\left[G{(X^n,Y^n|U,V)}^{\rho }\right]& \ge 2^{\rho (H_{\tilde{\rho }}(X^n,Y^n)-log|\mathcal{U}\times \mathcal{V}|-log{\tau }_n)}\hfill \end{array}$$

(59)

$$\begin{array}{cc}\hfill & \ge 2^{\rho n(\frac{1}{n}{H}_{\tilde{\rho }}(X^n,Y^n)-{\mathsf{R}}_{\mathsf{X}}-{\mathsf{R}}_{\mathsf{Y}}-\frac{1}{n}log{\tau }_n)}.\hfill \end{array}$$

(60)

It follows from (60), (58), and the fact that $\frac{1}{n}log{\tau }_n$ tends to zero as n tends to infinity that the LHS of (59) cannot tend to one as n tends to infinity, so $({\mathsf{R}}_{\mathsf{X}},{\mathsf{R}}_{\mathsf{Y}})$ is not achievable if (8) does not hold. The necessity of (6) and (7) can be shown in the same way. ☐

5. Achievability

In this section, we prove a nonasymptotic and an asymptotic achievability result (Theorem 3 and Corollary 2, respectively).

Theorem 3.

Let $\mathcal{X}$, $\mathcal{Y}$, $\mathcal{U}$, and $\mathcal{V}$ be finite nonempty sets; let $P_{XY}$ be a PMF; let $\rho >0$; and let $ϵ>0$ be such that:

$$\begin{array}{cc}\hfill log\left|\mathcal{U}\right|& \ge H_{\tilde{\rho }}\left(X\right|Y)+ϵ,\hfill \end{array}$$

(61)

$$\begin{array}{cc}\hfill log\left|\mathcal{V}\right|& \ge H_{\tilde{\rho }}\left(Y\right|X)+ϵ,\hfill \end{array}$$

(62)

$$\begin{array}{cc}\hfill log|\mathcal{U}\times \mathcal{V}|& \ge H_{\tilde{\rho }}(X,Y)+ϵ.\hfill \end{array}$$

(63)

Then, there exist functions $f:\mathcal{X}\to \mathcal{U}$ and $g:\mathcal{Y}\to \mathcal{V}$ and a guesser such that the ρth moment of the number of guesses needed to guess the pair $(X,Y)$ based on the side information $\left(f\right(X),g(Y\left)\right)$ satisfies:

$$\begin{array}{cc}\hfill \mathrm{E}\left[G{\left(X,Y\left|f\right(X),g(Y)\right)}^{\rho }\right]& \le \left\{\begin{array}{cc}1+4^{\rho +1}·2^{-\rho ϵ}\hfill & \mathrm{if}\rho \in (0,1],\hfill \\ 1+4^{{(\rho +1)}^2}·2^{-ϵ}\hfill & \mathrm{if}\rho >1.\hfill \end{array}\right.\hfill \end{array}$$

(64)

Proof. 

Our achievability result relies on random binning: we map each $x\in \mathcal{X}$ uniformly at random to some $u\in \mathcal{U}$ and each $y\in \mathcal{Y}$ uniformly at random to some $v\in \mathcal{V}$. We then show that the $\rho$th moment of the number of guesses averaged over all such mappings $f:\mathcal{X}\to \mathcal{U}$ and $g:\mathcal{Y}\to \mathcal{V}$ is upper bounded by the RHS of (64). From this, we conclude that there exist f and g that satisfy (64).

Let the guessing function G correspond to guessing in decreasing order of probability [2] (ties can be resolved arbitrarily). Let f and g be distributed as described above, and denote by ${\mathrm{E}}_{f,g}[·]$ the expectation with respect to f and g. Then,

$$\begin{array}{cc}\hfill {\mathrm{E}}_{f,g}\left[\mathrm{E}\left[G{(X,Y|f\left(X\right),g\left(Y\right))}^{\rho }\right]\right]& =\sum _{x,y}P(x,y){\mathrm{E}}_{f,g}\left[G{(x,y|f\left(x\right),g\left(y\right))}^{\rho }\right]\hfill \end{array}$$

(65)

$$\begin{array}{cc}\hfill & \le \sum _{x,y}P(x,y){\mathrm{E}}_{f,g}\left[{\left\{\sum _{x^{\prime },y^{\prime }}\psi (x^{\prime },y^{\prime }){\varphi }_f\left(x^{\prime }\right){\varphi }_g\left(y^{\prime }\right)\right\}}^{\rho }\right]\hfill \end{array}$$

(66)

$$\begin{array}{cc}\hfill & =\sum _{x,y}P(x,y){\mathrm{E}}_{f,g}\left[{(1+{\beta }_1+{\beta }_2+{\beta }_3)}^{\rho }\right]\hfill \end{array}$$

(67)

$$\begin{array}{cc}\hfill & \le 1+4^{\rho }\sum _{x,y}P(x,y)({\mathrm{E}}_{f,g}\left[{\beta }_1^{\rho }\right]+{\mathrm{E}}_{f,g}\left[{\beta }_2^{\rho }\right]+{\mathrm{E}}_{f,g}\left[{\beta }_3^{\rho }\right])\hfill \end{array}$$

(68)

with:

$$\begin{array}{cc}\hfill \psi (x^{\prime },y^{\prime })=\psi (x,y,x^{\prime },y^{\prime })& \triangleq 𝟙\{P(x^{\prime },y^{\prime })\ge P(x,y)\},\hfill \end{array}$$

(69)

$$\begin{array}{cc}\hfill {\varphi }_f\left(x^{\prime }\right)={\varphi }_f(x,x^{\prime })& \triangleq 𝟙\{f\left(x^{\prime }\right)=f\left(x\right)\},\hfill \end{array}$$

(70)

$$\begin{array}{cc}\hfill {\varphi }_g\left(y^{\prime }\right)={\varphi }_g(y,y^{\prime })& \triangleq 𝟙\{g\left(y^{\prime }\right)=g\left(y\right)\},\hfill \end{array}$$

(71)

$$\begin{array}{cc}\hfill {\beta }_1={\beta }_1(x,y,f)& \triangleq \sum _{x^{\prime }\ne x}\psi (x^{\prime },y){\varphi }_f\left(x^{\prime }\right),\hfill \end{array}$$

(72)

$$\begin{array}{cc}\hfill {\beta }_2={\beta }_2(x,y,g)& \triangleq \sum _{y^{\prime }\ne y}\psi (x,y^{\prime }){\varphi }_g\left(y^{\prime }\right),\hfill \end{array}$$

(73)

$$\begin{array}{cc}\hfill {\beta }_3={\beta }_3(x,y,f,g)& \triangleq \sum _{x^{\prime }\ne x,y^{\prime }\ne y}\psi (x^{\prime },y^{\prime }){\varphi }_f\left(x^{\prime }\right){\varphi }_g\left(y^{\prime }\right),\hfill \end{array}$$

(74)

where $𝟙\{·\}$ is the indicator function that is one if the condition comprising its argument is true and zero otherwise; (65) holds because $(f,g)$ and $(X,Y)$ are independent; (66) holds because the number of guesses is upper bounded by the number of $(x^{\prime },y^{\prime })$ that are at least as likely as $(x,y)$ and that are mapped to the same labels $(u,v)$ as $(x,y)$; (67) follows from splitting the sum depending on whether $x^{\prime }=x$ or not and whether $y^{\prime }=y$ or not and from the fact that $\psi (x,y)={\varphi }_f\left(x\right)={\varphi }_g\left(y\right)=1$; and (68) follows from Lemma 5 because ${\beta }_1$, ${\beta }_2$, and ${\beta }_3$ are nonnegative integers. As indicated in (69)–(74), the dependence of $\psi$, ${\varphi }_f$, ${\varphi }_g$, ${\beta }_1$, ${\beta }_2$, and ${\beta }_3$ on x, y, f, and g is implicit in our notation.

We first treat the case $\rho \in (0,1]$. We bound the terms on the RHS of (68) as follows:

$$\begin{array}{cc}\hfill \sum _{x,y}P(x,y){\mathrm{E}}_{f,g}\left[{\beta }_1^{\rho }\right]& \le \sum _{x,y}P(x,y){\mathrm{E}}_{f,g}{\left[{\beta }_1\right]}^{\rho }\hfill \end{array}$$

(75)

$$\begin{array}{cc}\hfill & =\sum _{x,y}P(x,y){\left[\sum _{x^{\prime }\ne x}\psi (x^{\prime },y)\frac{1}{\left|\mathcal{U}\right|}\right]}^{\rho }\hfill \end{array}$$

(76)

$$\begin{array}{cc}\hfill & \le \sum _{x,y}P(x,y){\left[\sum _{x^{\prime }}{\left[\frac{P(x^{\prime },y)}{P(x,y)}\right]}^{\tilde{\rho }}\frac{1}{\left|\mathcal{U}\right|}\right]}^{\rho }\hfill \end{array}$$

(77)

$$\begin{array}{c}=\frac{1}{{\left|\mathcal{U}\right|}^{\rho }}\sum _{x,y}P{(x,y)}^{\tilde{\rho }}{\left[\sum _{x^{\prime }}P{(x^{\prime },y)}^{\tilde{\rho }}\right]}^{\rho }\hfill \end{array}$$

(78)

$$\begin{array}{c}=\frac{1}{{\left|\mathcal{U}\right|}^{\rho }}\sum _y\left[\sum _{x}P{(x,y)}^{\tilde{\rho }}\right]{\left[\sum _{x^{\prime }}P{(x^{\prime },y)}^{\tilde{\rho }}\right]}^{\rho }\hfill \end{array}$$

(79)

$$\begin{array}{c}=\frac{1}{{\left|\mathcal{U}\right|}^{\rho }}\sum _y{\left[\sum _{x}P{(x,y)}^{\tilde{\rho }}\right]}^{1+\rho }\hfill \end{array}$$

(80)

$$\begin{array}{cc}\hfill & =2^{\rho (H_{\tilde{\rho }}\left(X\right|Y)-log\left|\mathcal{U}\right|)}\hfill \end{array}$$

(81)

$$\begin{array}{cc}\hfill & \le 2^{-\rho ϵ},\hfill \end{array}$$

(82)

where (75) follows from Jensen’s inequality because $z\mapsto z^{\rho }$ is concave on ${\mathbb{R}}_{\ge 0}$ since $\rho \in (0,1]$; (76) holds because the expectation operator is linear and because ${\mathrm{E}}_{f,g}\left[{\varphi }_f\left(x^{\prime }\right)\right]=1/\left|\mathcal{U}\right|$ since $x^{\prime }\ne x$; in (77), we extended the inner summation and used that $\psi (x^{\prime },y)\le {[P(x^{\prime },y)/P(x,y)]}^{\tilde{\rho }}$; and (82) follows from (61). In the same way, we obtain:

$$\begin{array}{c}\hfill \sum _{x,y}P(x,y){\mathrm{E}}_{f,g}\left[{\beta }_2^{\rho }\right]\le 2^{-\rho ϵ}.\end{array}$$

(83)

Similarly,

$$\begin{array}{cc}\hfill \sum _{x,y}P(x,y){\mathrm{E}}_{f,g}\left[{\beta }_3^{\rho }\right]& \le \sum _{x,y}P(x,y){\mathrm{E}}_{f,g}{\left[{\beta }_3\right]}^{\rho }\hfill \end{array}$$

(84)

$$\begin{array}{cc}\hfill & =\sum _{x,y}P(x,y){\left[\sum _{x^{\prime }\ne x,y^{\prime }\ne y}\psi (x^{\prime },y^{\prime })\frac{1}{|\mathcal{U}\times \mathcal{V}|}\right]}^{\rho }\hfill \end{array}$$

(85)

$$\begin{array}{c}\le \sum _{x,y}P(x,y){\left[\sum _{x^{\prime },y^{\prime }}{\left[\frac{P(x^{\prime },y^{\prime })}{P(x,y)}\right]}^{\tilde{\rho }}\frac{1}{|\mathcal{U}\times \mathcal{V}|}\right]}^{\rho }\hfill \end{array}$$

(86)

$$\begin{array}{c}=\frac{1}{{|\mathcal{U}\times \mathcal{V}|}^{\rho }}\sum _{x,y}P{(x,y)}^{\tilde{\rho }}{\left[\sum _{x^{\prime },y^{\prime }}P{(x^{\prime },y^{\prime })}^{\tilde{\rho }}\right]}^{\rho }\hfill \end{array}$$

(87)

$$\begin{array}{c}=\frac{1}{{|\mathcal{U}\times \mathcal{V}|}^{\rho }}{\left[\sum _{x,y}P{(x,y)}^{\tilde{\rho }}\right]}^{1+\rho }\hfill \end{array}$$

(88)

$$\begin{array}{cc}\hfill & =2^{\rho (H_{\tilde{\rho }}(X,Y)-log|\mathcal{U}\times \mathcal{V}|)}\hfill \end{array}$$

(89)

$$\begin{array}{cc}\hfill & \le 2^{-\rho ϵ}.\hfill \end{array}$$

(90)

From (68), (82), (83), and (90), we obtain:

$$\begin{array}{cc}\hfill {\mathrm{E}}_{f,g}\left[\mathrm{E}\left[G{(X,Y|f\left(X\right),g\left(Y\right))}^{\rho }\right]\right]& \le 1+3·4^{\rho }·2^{-\rho ϵ}\hfill \end{array}$$

(91)

$$\begin{array}{c}\le 1+4^{\rho +1}·2^{-\rho ϵ}\hfill \end{array}$$

(92)

and hence infer the existence of $f: \mathcal{X}\to \mathcal{U}$ and $g: \mathcal{Y}\to \mathcal{V}$ satisfying (64).

We now consider (68) when $\rho >1$. Unlike in the case $\rho \in (0,1]$, we cannot use Jensen’s inequality as we did in (75). Instead, for fixed $x\in \mathcal{X}$ and $y\in \mathcal{Y}$, we upper-bound the first expectation on the RHS of (68) by:

$$\begin{array}{cc}\hfill {\mathrm{E}}_{f,g}\left[{\beta }_1^{\rho }\right]& \le 2^{{\rho }^2}max\left\{{\mathrm{E}}_{f,g}\left[{\beta }_1\right],{\mathrm{E}}_{f,g}{\left[{\beta }_1\right]}^{\rho }\right\}\hfill \end{array}$$

(93)

$$\begin{array}{cc}\hfill & \le 2^{{\rho }^2}\left({\mathrm{E}}_{f,g}{\left[{\beta }_1\right]}^{\rho }+{\mathrm{E}}_{f,g}\left[{\beta }_1\right]\right),\hfill \end{array}$$

(94)

where (93) follows from Lemma 7 because $\rho >1$ and because ${\beta }_1$ is a sum of independent random variables taking values in $\{0,1\}$. By the same steps as in (76)–(82),

$$\begin{array}{c}\hfill \sum _{x,y}P(x,y){\mathrm{E}}_{f,g}{\left[{\beta }_1\right]}^{\rho }\le 2^{-\rho ϵ}.\end{array}$$

(95)

As to the expectation of the other term on the RHS of (94),

$$\begin{array}{cc}\hfill \sum _{x,y}P(x,y){\mathrm{E}}_{f,g}\left[{\beta }_1\right]& \le {\left[\sum _{x,y}P(x,y){\mathrm{E}}_{f,g}{\left[{\beta }_1\right]}^{\rho }\right]}^{\frac{1}{\rho }}\hfill \end{array}$$

(96)

$$\begin{array}{cc}\hfill & \le 2^{-ϵ},\hfill \end{array}$$

(97)

where (96) follows from Jensen’s inequality because $z\mapsto z^{\frac{1}{\rho }}$ is concave on ${\mathbb{R}}_{\ge 0}$ since $\rho >1$, and (97) follows from (95). From (94), (95), and (97), we obtain:

$$\begin{array}{cc}\hfill \sum _{x,y}P(x,y){\mathrm{E}}_{f,g}\left[{\beta }_1^{\rho }\right]& \le 2^{{\rho }^2}(2^{-\rho ϵ}+2^{-ϵ})\hfill \end{array}$$

(98)

$$\begin{array}{c}\le 2^{{\rho }^2+1}·2^{-ϵ},\hfill \end{array}$$

(99)

where (99) holds because $2^{-\rho ϵ}\le 2^{-ϵ}$ since $\rho >1$ and $ϵ>0$. In the same way, we obtain for the second expectation on the RHS of (68):

$$\begin{array}{c}\hfill \sum _{x,y}P(x,y){\mathrm{E}}_{f,g}\left[{\beta }_2^{\rho }\right]\le 2^{{\rho }^2+1}·2^{-ϵ}.\end{array}$$

(100)

Bounding ${\mathrm{E}}_{f,g}\left[{\beta }_3^{\rho }\right]$, i.e., the third expectation on the RHS of (68), is more involved because ${\beta }_3$ is not a sum of independent random variables. Our approach builds on the ideas used by Rosenthal [11] (Proof of Lemma 1); compare (47) and (48) with (108) and (123) ahead. For fixed $x\in \mathcal{X}$ and $y\in \mathcal{Y}$,

$$\begin{array}{cc}\hfill {\mathrm{E}}_{f,g}\left[{\beta }_3^{\rho }\right]& ={\mathrm{E}}_{f,g}\left[\sum _{x^{\prime }\ne x,y^{\prime }\ne y}\psi (x^{\prime },y^{\prime }){\varphi }_f\left(x^{\prime }\right){\varphi }_g\left(y^{\prime }\right)·{\left\{\sum _{\tilde{x}\ne x,\tilde{y}\ne y}\psi (\tilde{x},\tilde{y}){\varphi }_f\left(\tilde{x}\right){\varphi }_g\left(\tilde{y}\right)\right\}}^{\rho -1}\right]\hfill \end{array}$$

(101)

$$\begin{array}{c}={\mathrm{E}}_{f,g}\left[\sum _{x^{\prime }\ne x,y^{\prime }\ne y}\psi (x^{\prime },y^{\prime }){\varphi }_f\left(x^{\prime }\right){\varphi }_g\left(y^{\prime }\right)·{(1+{\gamma }_1+{\gamma }_2+{\gamma }_3)}^{\rho -1}\right]\hfill \end{array}$$

(102)

$$\begin{array}{c}=\sum _{x^{\prime }\ne x,y^{\prime }\ne y}{\mathrm{E}}_{f,g}\left[\psi (x^{\prime },y^{\prime }){\varphi }_f\left(x^{\prime }\right){\varphi }_g\left(y^{\prime }\right)·{(1+{\gamma }_1+{\gamma }_2+{\gamma }_3)}^{\rho -1}\right]\hfill \end{array}$$

(103)

$$\begin{array}{cc}\hfill & =\sum _{x^{\prime }\ne x,y^{\prime }\ne y}{\mathrm{E}}_{f,g}\left[\psi (x^{\prime },y^{\prime }){\varphi }_f\left(x^{\prime }\right){\varphi }_g\left(y^{\prime }\right)\right]·{\mathrm{E}}_{f,g}\left[{(1+{\gamma }_1+{\gamma }_2+{\gamma }_3)}^{\rho -1}\right]\hfill \end{array}$$

(104)

$$\begin{array}{c}\le \sum _{x^{\prime }\ne x,y^{\prime }\ne y}{\mathrm{E}}_{f,g}\left[\psi (x^{\prime },y^{\prime }){\varphi }_f\left(x^{\prime }\right){\varphi }_g\left(y^{\prime }\right)\right]·{\mathrm{E}}_{f,g}\left[{(1+{\delta }_1+{\delta }_2+{\beta }_3)}^{\rho -1}\right]\hfill \end{array}$$

(105)

$$\begin{array}{cc}\hfill & \le \sum _{x^{\prime }\ne x,y^{\prime }\ne y}{\mathrm{E}}_{f,g}\left[\psi (x^{\prime },y^{\prime }){\varphi }_f\left(x^{\prime }\right){\varphi }_g\left(y^{\prime }\right)\right]·4^{\rho -1}·{\mathrm{E}}_{f,g}\left[1+{\delta }_1^{\rho -1}+{\delta }_2^{\rho -1}+{\beta }_3^{\rho -1}\right]\hfill \end{array}$$

(106)

$$\begin{array}{c}=4^{\rho -1}\{{\mathrm{E}}_{f,g}\left[{\beta }_3\right]+\sum _{y^{\prime }\ne y}\frac{1}{\left|\mathcal{V}\right|}{\mathrm{E}}_{f,g}\left[{\delta }_1\right]{\mathrm{E}}_{f,g}\left[{\delta }_1^{\rho -1}\right]\hfill \end{array}$$

$$\begin{array}{cc}\hfill & \phantom{=}\phantom{4^{\rho -1}\{}+\sum _{x^{\prime }\ne x}\frac{1}{\left|\mathcal{U}\right|}{\mathrm{E}}_{f,g}\left[{\delta }_2\right]{\mathrm{E}}_{f,g}\left[{\delta }_2^{\rho -1}\right]+{\mathrm{E}}_{f,g}\left[{\beta }_3\right]{\mathrm{E}}_{f,g}\left[{\beta }_3^{\rho -1}\right]\}\hfill \end{array}$$

(107)

$$\begin{array}{c}\le 4^{\rho }max\{{\mathrm{E}}_{f,g}\left[{\beta }_3\right],\sum _{y^{\prime }\ne y}\frac{1}{\left|\mathcal{V}\right|}{\mathrm{E}}_{f,g}\left[{\delta }_1\right]{\mathrm{E}}_{f,g}\left[{\delta }_1^{\rho -1}\right],\hfill \end{array}$$

$$\begin{array}{c}\phantom{\le }\phantom{4^{\rho }max\{}\sum _{x^{\prime }\ne x}\frac{1}{\left|\mathcal{U}\right|}{\mathrm{E}}_{f,g}\left[{\delta }_2\right]{\mathrm{E}}_{f,g}\left[{\delta }_2^{\rho -1}\right],{\mathrm{E}}_{f,g}\left[{\beta }_3\right]{\mathrm{E}}_{f,g}\left[{\beta }_3^{\rho -1}\right]\}\hfill \end{array}$$

(108)

with:

$$\begin{array}{cc}\hfill {\gamma }_1={\gamma }_1(x,y,x^{\prime },y^{\prime },f)& \triangleq \sum _{\tilde{x}\notin \{x,x^{\prime }\}}\psi (\tilde{x},y^{\prime }){\varphi }_f\left(\tilde{x}\right),\hfill \end{array}$$

(109)

$$\begin{array}{cc}\hfill {\gamma }_2={\gamma }_2(x,y,x^{\prime },y^{\prime },g)& \triangleq \sum _{\tilde{y}\notin \{y,y^{\prime }\}}\psi (x^{\prime },\tilde{y}){\varphi }_g\left(\tilde{y}\right),\hfill \end{array}$$

(110)

$$\begin{array}{cc}\hfill {\gamma }_3={\gamma }_3(x,y,x^{\prime },y^{\prime },f,g)& \triangleq \sum _{\tilde{x}\notin \{x,x^{\prime }\},\tilde{y}\notin \{y,y^{\prime }\}}\psi (\tilde{x},\tilde{y}){\varphi }_f\left(\tilde{x}\right){\varphi }_g\left(\tilde{y}\right),\hfill \end{array}$$

(111)

$$\begin{array}{cc}\hfill {\delta }_1={\delta }_1(x,y,y^{\prime },f)& \triangleq \sum _{\tilde{x}\ne x}\psi (\tilde{x},y^{\prime }){\varphi }_f\left(\tilde{x}\right),\hfill \end{array}$$

(112)

$$\begin{array}{cc}\hfill {\delta }_2={\delta }_2(x,y,x^{\prime },g)& \triangleq \sum _{\tilde{y}\ne y}\psi (x^{\prime },\tilde{y}){\varphi }_g\left(\tilde{y}\right),\hfill \end{array}$$

(113)

where (102) follows from splitting the sum in braces depending on whether $\tilde{x}=x^{\prime }$ or not and whether $\tilde{y}=y^{\prime }$ or not and from assuming $\psi (x^{\prime },y^{\prime })={\varphi }_f\left(x^{\prime }\right)={\varphi }_g\left(y^{\prime }\right)=1$ within the braces, which does not change the value of the expression because it is multiplied by $\psi (x^{\prime },y^{\prime }){\varphi }_f\left(x^{\prime }\right){\varphi }_g\left(y^{\prime }\right)$; (104) holds because $({\varphi }_f\left(x^{\prime }\right),{\varphi }_g\left(y^{\prime }\right))$ and $({\gamma }_1,{\gamma }_2,{\gamma }_3)$ are independent since $\tilde{x}\ne x^{\prime }$ and $\tilde{y}\ne y^{\prime }$; (105) holds because $\rho -1>0$, ${\gamma }_1\le {\delta }_1$, ${\gamma }_2\le {\delta }_2$, and ${\gamma }_3\le {\beta }_3$; (106) follows from Lemma 6; and (107) follows from identifying ${\mathrm{E}}_{f,g}\left[{\beta }_3\right]$, ${\mathrm{E}}_{f,g}\left[{\delta }_1\right]$, and ${\mathrm{E}}_{f,g}\left[{\delta }_2\right]$ because ${\varphi }_f\left(x^{\prime }\right)$ and ${\varphi }_g\left(y^{\prime }\right)$ are independent, ${\mathrm{E}}_{f,g}\left[{\varphi }_f\left(x^{\prime }\right)\right]=1/\left|\mathcal{U}\right|$, and ${\mathrm{E}}_{f,g}\left[{\varphi }_g\left(y^{\prime }\right)\right]=1/\left|\mathcal{V}\right|$. As indicated in (109)–(113), the dependence of ${\gamma }_1$, ${\gamma }_2$, ${\gamma }_3$, ${\delta }_1$, and ${\delta }_2$ on x, y, $x^{\prime }$, $y^{\prime }$, f, and g is implicit in our notation.

To bound ${\mathrm{E}}_{f,g}\left[{\beta }_3^{\rho }\right]$ further, we study some of the terms on the RHS of (108) separately, starting with the second, which involves the sum over $y^{\prime }$. For fixed $x\in \mathcal{X}$, $y\in \mathcal{Y}$, and $y^{\prime }\in \mathcal{Y}\setminus \left\{y\right\}$,

$$\begin{array}{cc}\hfill {\mathrm{E}}_{f,g}\left[{\delta }_1\right]{\mathrm{E}}_{f,g}\left[{\delta }_1^{\rho -1}\right]& \le {\mathrm{E}}_{f,g}{\left[{\delta }_1^{\rho }\right]}^{\frac{1}{\rho }}{\mathrm{E}}_{f,g}{\left[{\delta }_1^{\rho }\right]}^{\frac{\rho -1}{\rho }}\hfill \end{array}$$

(114)

$$\begin{array}{c}={\mathrm{E}}_{f,g}\left[{\delta }_1^{\rho }\right]\hfill \end{array}$$

(115)

$$\begin{array}{cc}\hfill & \le 2^{{\rho }^2}max\left\{{\mathrm{E}}_{f,g}\left[{\delta }_1\right],{\mathrm{E}}_{f,g}{\left[{\delta }_1\right]}^{\rho }\right\}\hfill \end{array}$$

(116)

$$\begin{array}{c}\le 2^{{\rho }^2}\left({\mathrm{E}}_{f,g}\left[{\delta }_1\right]+{\mathrm{E}}_{f,g}{\left[{\delta }_1\right]}^{\rho }\right),\hfill \end{array}$$

(117)

where (114) follows from Jensen’s inequality because $z\mapsto z^{\frac{1}{\rho }}$ and $z\mapsto z^{\frac{\rho -1}{\rho }}$ are both concave on ${\mathbb{R}}_{\ge 0}$ since $\rho >1$, and (116) follows from Lemma 7 because $\rho >1$ and because ${\delta }_1$ is a sum of independent random variables taking values in $\{0,1\}$. This implies that for fixed $x\in \mathcal{X}$ and $y\in \mathcal{Y}$,

$$\begin{array}{cc}\hfill \sum _{y^{\prime }\ne y}\frac{1}{\left|\mathcal{V}\right|}{\mathrm{E}}_{f,g}\left[{\delta }_1\right]{\mathrm{E}}_{f,g}\left[{\delta }_1^{\rho -1}\right]& \le 2^{{\rho }^2}\sum _{y^{\prime }\ne y}\frac{1}{\left|\mathcal{V}\right|}\left({\mathrm{E}}_{f,g}\left[{\delta }_1\right]+{\mathrm{E}}_{f,g}{\left[{\delta }_1\right]}^{\rho }\right)\hfill \end{array}$$

(118)

$$\begin{array}{cc}\hfill & =2^{{\rho }^2}{\mathrm{E}}_{f,g}\left[{\beta }_3\right]+2^{{\rho }^2}\sum _{y^{\prime }\ne y}\frac{1}{\left|\mathcal{V}\right|}{\mathrm{E}}_{f,g}{\left[{\delta }_1\right]}^{\rho },\hfill \end{array}$$

(119)

where (119) follows from the definitions of ${\delta }_1$ and ${\beta }_3$. Similarly, for the third term on the RHS of (108),

$$\begin{array}{cc}\hfill \sum _{x^{\prime }\ne x}\frac{1}{\left|\mathcal{U}\right|}{\mathrm{E}}_{f,g}\left[{\delta }_2\right]{\mathrm{E}}_{f,g}\left[{\delta }_2^{\rho -1}\right]& \le 2^{{\rho }^2}{\mathrm{E}}_{f,g}\left[{\beta }_3\right]+2^{{\rho }^2}\sum _{x^{\prime }\ne x}\frac{1}{\left|\mathcal{U}\right|}{\mathrm{E}}_{f,g}{\left[{\delta }_2\right]}^{\rho }.\hfill \end{array}$$

(120)

With the help of (119) and (120), we now go back to (108) and argue that it implies that for fixed $x\in \mathcal{X}$ and $y\in \mathcal{Y}$,

$${\mathrm{E}}_{f,g}\left[{\beta }_3^{\rho }\right]\le 2·4^{{\rho }^2} [{\mathrm{E}}_{f,g}\left[{\beta }_3\right]+\sum _{y^{\prime }\ne y}\frac{1}{\left|\mathcal{V}\right|}{\mathrm{E}}_{f,g}{\left[{\delta }_1\right]}^{\rho }+\sum _{x^{\prime }\ne x}\frac{1}{\left|\mathcal{U}\right|}{\mathrm{E}}_{f,g}{\left[{\delta }_2\right]}^{\rho }+{\mathrm{E}}_{f,g}{\left[{\beta }_3\right]}^{\rho }].$$

(121)

To prove this, we consider four cases depending on which term on the RHS of (108) achieves the maximum: If ${\mathrm{E}}_{f,g}\left[{\beta }_3\right]$ achieves the maximum, then (121) holds because $4^{\rho }\le 2·4^{{\rho }^2}$. If the LHS of (118) achieves the maximum, then (121) follows from (119) because $4^{\rho }·2^{{\rho }^2}\le 2·4^{{\rho }^2}$. If the LHS of (120) achieves the maximum, then (121) follows similarly. Finally, if ${\mathrm{E}}_{f,g}\left[{\beta }_3\right]{\mathrm{E}}_{f,g}\left[{\beta }_3^{\rho -1}\right]$ achieves the maximum, then:

$$\begin{array}{cc}\hfill {\mathrm{E}}_{f,g}\left[{\beta }_3^{\rho }\right]& \le 4^{\rho }{\mathrm{E}}_{f,g}\left[{\beta }_3\right]{\mathrm{E}}_{f,g}\left[{\beta }_3^{\rho -1}\right]\hfill \end{array}$$

(122)

$$\begin{array}{c}\le 4^{\rho }{\mathrm{E}}_{f,g}\left[{\beta }_3\right]{\mathrm{E}}_{f,g}{\left[{\beta }_3^{\rho }\right]}^{\frac{\rho -1}{\rho }},\hfill \end{array}$$

(123)

where (123) follows from Jensen’s inequality because $z\mapsto z^{\frac{\rho -1}{\rho }}$ is concave on ${\mathbb{R}}_{\ge 0}$ for $\rho >1$. Rearranging (123), we obtain:

$$\begin{array}{c}\hfill {\mathrm{E}}_{f,g}\left[{\beta }_3^{\rho }\right]\le 4^{{\rho }^2}{\mathrm{E}}_{f,g}{\left[{\beta }_3\right]}^{\rho },\end{array}$$

(124)

so (121) holds also in this case.

Having established (121), we now take the expectation of its sides to obtain:

$$\begin{array}{c}\sum _{x,y}P(x,y){\mathrm{E}}_{f,g}\left[{\beta }_3^{\rho }\right]\le 2·4^{{\rho }^2}\sum _{x,y}P(x,y)\left[{\mathrm{E}}_{f,g}\left[{\beta }_3\right]+\sum _{y^{\prime }\ne y}\frac{1}{\left|\mathcal{V}\right|}{\mathrm{E}}_{f,g}{\left[{\delta }_1\right]}^{\rho }+\sum _{x^{\prime }\ne x}\frac{1}{\left|\mathcal{U}\right|}{\mathrm{E}}_{f,g}{\left[{\delta }_2\right]}^{\rho }+{\mathrm{E}}_{f,g}{\left[{\beta }_3\right]}^{\rho }\right].\end{array}$$

(125)

We now study the terms on the RHS of (125) separately, starting with the fourth (last). By (85)–(90), which hold also if $\rho >1$,

$$\begin{array}{c}\hfill \sum _{x,y}P(x,y){\mathrm{E}}_{f,g}{\left[{\beta }_3\right]}^{\rho }\le 2^{-\rho ϵ}.\end{array}$$

(126)

As for the first term on the RHS of (125),

$$\begin{array}{c}\hfill \sum _{x,y}P(x,y){\mathrm{E}}_{f,g}\left[{\beta }_3\right]\le 2^{-ϵ},\end{array}$$

(127)

which follows from (126) in the same way as (97) followed from (95). As for the second term on the RHS of (125),

$$\begin{array}{cc}\hfill \sum _{x,y}P& (x,y)\sum _{y^{\prime }\ne y}\frac{1}{\left|\mathcal{V}\right|}{\mathrm{E}}_{f,g}{\left[{\delta }_1\right]}^{\rho }\hfill \end{array}$$

$$\begin{array}{c}=\sum _{x,y}P(x,y)\sum _{y^{\prime }\ne y}\frac{1}{\left|\mathcal{V}\right|}{\left[\sum _{x^{\prime }\ne x}\psi (x^{\prime },y^{\prime })\frac{1}{\left|\mathcal{U}\right|}\right]}^{\rho }\hfill \end{array}$$

(128)

$$\begin{array}{cc}\hfill & \le \sum _{x,y}P{(x,y)}^{\tilde{\rho }}\sum _{y^{\prime }}\frac{1}{\left|\mathcal{V}\right|}{\left[\sum _{x^{\prime }}P{(x^{\prime },y^{\prime })}^{\tilde{\rho }}\frac{1}{\left|\mathcal{U}\right|}\right]}^{\rho }\hfill \end{array}$$

(129)

$$\begin{array}{c}=\sum _{x,y}P{(x,y)}^{\tilde{\rho }}\sum _{y^{\prime }}{\left[\sum _{x^{\prime }}P{(x^{\prime },y^{\prime })}^{\tilde{\rho }}\frac{1}{{|\mathcal{U}\times \mathcal{V}|}^{\rho }}\right]}^{\frac{1}{\rho }}·{\left[\sum _{x^{\prime }}P{(x^{\prime },y^{\prime })}^{\tilde{\rho }}\frac{1}{{\left|\mathcal{U}\right|}^{\frac{\rho }{1+\rho }}}\right]}^{(1+\rho )·\frac{\rho -1}{\rho }}\hfill \end{array}$$

(130)

$$\begin{array}{cc}\hfill & \le \sum _{x,y}P{(x,y)}^{\tilde{\rho }}{\left\{\sum _{y^{\prime }}\sum _{x^{\prime }}P{(x^{\prime },y^{\prime })}^{\tilde{\rho }}\frac{1}{{|\mathcal{U}\times \mathcal{V}|}^{\rho }}\right\}}^{\frac{1}{\rho }}·{\left\{\sum _{y^{\prime }}{\left[\sum _{x^{\prime }}P{(x^{\prime },y^{\prime })}^{\tilde{\rho }}\frac{1}{{\left|\mathcal{U}\right|}^{\frac{\rho }{1+\rho }}}\right]}^{1+\rho }\right\}}^{\frac{\rho -1}{\rho }}\hfill \end{array}$$

(131)

$$\begin{array}{c}={\left\{\frac{1}{{|\mathcal{U}\times \mathcal{V}|}^{\rho }}{\left[\sum _{x,y}P{(x,y)}^{\tilde{\rho }}\right]}^{1+\rho }\right\}}^{\frac{1}{\rho }}·{\left\{\frac{1}{{\left|\mathcal{U}\right|}^{\rho }}\sum _{y^{\prime }}{\left[\sum _{x^{\prime }}P{(x^{\prime },y^{\prime })}^{\tilde{\rho }}\right]}^{1+\rho }\right\}}^{\frac{\rho -1}{\rho }}\hfill \end{array}$$

(132)

$$\begin{array}{cc}\hfill & \le {\left(2^{-\rho ϵ}\right)}^{\frac{1}{\rho }}·{\left(2^{-\rho ϵ}\right)}^{\frac{\rho -1}{\rho }}\hfill \end{array}$$

(133)

$$\begin{array}{cc}\hfill & =2^{-\rho ϵ},\hfill \end{array}$$

(134)

where in (129), we extended the inner summations and used that $\psi (x^{\prime },y^{\prime })\le {[P(x^{\prime },y^{\prime })/P(x,y)]}^{\tilde{\rho }}$; (131) follows from Hölder’s inequality; and (133) follows from (89)–(90) and (81)–(82). In the same way, we obtain for the third term on the RHS of (125):

$$\begin{array}{c}\hfill \sum _{x,y}P(x,y)\sum _{x^{\prime }\ne x}\frac{1}{\left|\mathcal{U}\right|}{\mathrm{E}}_{f,g}{\left[{\delta }_2\right]}^{\rho }\le 2^{-\rho ϵ}.\end{array}$$

(135)

From (125), (127), (134), (135), and (126), we deduce:

$$\begin{array}{cc}\hfill \sum _{x,y}P(x,y){\mathrm{E}}_{f,g}\left[{\beta }_3^{\rho }\right]& \le 2·4^{{\rho }^2}(2^{-ϵ}+2^{-\rho ϵ}+2^{-\rho ϵ}+2^{-\rho ϵ})\hfill \end{array}$$

(136)

$$\begin{array}{c}\le 8·4^{{\rho }^2}·2^{-ϵ},\hfill \end{array}$$

(137)

where (137) holds because $2^{-\rho ϵ}\le 2^{-ϵ}$ since $\rho >1$ and $ϵ>0$. Finally, (68), (99), (100), and (137) imply:

$$\begin{array}{cc}\hfill {\mathrm{E}}_{f,g}\left[\mathrm{E}\left[G{(X,Y|f\left(X\right),g\left(Y\right))}^{\rho }\right]\right]& \le 1+4^{\rho }(2·2^{{\rho }^2+1}·2^{-ϵ}+8·4^{{\rho }^2}·2^{-ϵ})\hfill \end{array}$$

(138)

$$\begin{array}{c}\le 1+4^{{(\rho +1)}^2}·2^{-ϵ}\hfill \end{array}$$

(139)

and thus prove the existence of $f:\mathcal{X}\to \mathcal{U}$ and $g:\mathcal{Y}\to \mathcal{V}$ satisfying (64). ☐

Corollary 2.

For any $\rho >0$, rate pairs in the interior of $\mathcal{R}\left(\rho \right)$ are achievable.

Proof. 

Let $({\mathsf{R}}_{\mathsf{X}},{\mathsf{R}}_{\mathsf{Y}})$ be in the interior of $\mathcal{R}\left(\rho \right)$. Then, (6)–(8) hold with strict inequalities, and there exists a $\delta >0$ such that for all sufficiently large n,

$$\begin{array}{cc}\hfill log\lfloor 2^{n{\mathsf{R}}_{\mathsf{X}}}\rfloor & \ge H_{\tilde{\rho }}\left(X^n\right|Y^n)+n\delta ,\hfill \end{array}$$

(140)

$$\begin{array}{cc}\hfill log\lfloor 2^{n{\mathsf{R}}_{\mathsf{Y}}}\rfloor & \ge H_{\tilde{\rho }}\left(Y^n\right|X^n)+n\delta ,\hfill \end{array}$$

(141)

$$\begin{array}{cc}\hfill log\lfloor 2^{n{\mathsf{R}}_{\mathsf{X}}}\rfloor +log\lfloor 2^{n{\mathsf{R}}_{\mathsf{Y}}}\rfloor & \ge H_{\tilde{\rho }}(X^n,Y^n)+n\delta .\hfill \end{array}$$

(142)

Using Theorem 3 with ${\mathcal{X}}^{\prime }\triangleq {\mathcal{X}}^n$, ${\mathcal{Y}}^{\prime }\triangleq {\mathcal{Y}}^n$, $\mathcal{U}\triangleq \{1,\dots ,\lfloor 2^{n{\mathsf{R}}_{\mathsf{X}}}\rfloor \}$, $\mathcal{V}\triangleq \{1,\dots ,\lfloor 2^{n{\mathsf{R}}_{\mathsf{Y}}}\rfloor \}$, $P_{X^{\prime }{Y}^{\prime }}\triangleq P_{X^n{Y}^n}$, and ${ϵ}_n\triangleq n\delta$ shows that, for all sufficiently large n, there exist encoders $f_n:{\mathcal{X}}^n\to \mathcal{U}$ and $g_n:{\mathcal{Y}}^n\to \mathcal{V}$ and a guessing function $G_n$ satisfying:

$$\begin{array}{cc}\hfill \mathrm{E}\left[G_n{\left(X^n,Y^n|f_n\left(X^n\right),g_n\left(Y^n\right)\right)}^{\rho }\right]& \le \left\{\begin{array}{cc}1+4^{\rho +1}·2^{-\rho {ϵ}_n}\hfill & \mathrm{if}\rho \in (0,1],\hfill \\ 1+4^{{(\rho +1)}^2}·2^{-{ϵ}_n}\hfill & \mathrm{if}\rho >1.\hfill \end{array}\right.\hfill \end{array}$$

(143)

Because ${ϵ}_n$ tends to infinity as n tends to infinity, the RHS of (143) tends to one as n tends to infinity, which implies that the rate pair $({\mathsf{R}}_{\mathsf{X}},{\mathsf{R}}_{\mathsf{Y}})$ is achievable.  ☐