Information Bottleneck for a Rayleigh Fading MIMO Channel with an Oblivious Relay

Abstract

This paper considers the information bottleneck (IB) problem of a Rayleigh fading multiple-input multiple-out (MIMO) channel with an oblivious relay. The relay is constrained to operating without knowledge of the codebooks, i.e., it performs oblivious processing. Moreover, due to the bottleneck constraint, it is impossible for the relay to inform the destination node of the perfect channel state information (CSI) in each channel realization. To evaluate the bottleneck rate, we first provide an upper bound by assuming that the destination node can obtain a perfect CSI at no cost. Then, we provide four achievable schemes, where each scheme satisfies the bottleneck constraint and gives a lower bound to the bottleneck rate. In the first and second schemes, the relay splits the capacity of the relay–destination link into two parts and conveys both the CSI and its observation to the destination node. Due to CSI transmission, the performance of these two schemes is sensitive to the MIMO channel dimension, especially the channel input dimension. To ensure that it still performs well when the channel dimension grows large, in the third and fourth achievable schemes, the relay only transmits compressed observations to the destination node. Numerical results show that, with simple symbol-by-symbol oblivious relay processing and compression, the proposed achievable schemes work well and can demonstrate lower bounds that come quite close to the upper bound on a wide range of relevant system parameters.

1. Introduction

For a Markov chain $X\to Y\to Z$ and an assigned joint probability distribution $p_{X,Y}$, consider the following information bottleneck (IB) problem:

$$\begin{array}{cc}\hfill \underset{p_{Z|Y}}{max}& I(X;Z)\hfill \end{array}$$

(1a)

$$\begin{array}{cc}\hfill \mathrm{s}.\mathrm{t}.& I(Y;Z)\le C,\hfill \end{array}$$

(1b)

where C is the bottleneck constraint parameter and the optimization is with respect to the conditional probability distribution $p_{Z|Y}$ of Z given Y. Formulation (1) was introduced by Tishby in [1] and has found remarkable applications in supervised and unsupervised learning problems such as classification, clustering, prediction, etc. [2,3,4,5,6,7]. From a more fundamental information theoretic viewpoint, the IB arises from the classical remote source coding problem [8,9,10] under logarithmic distortion [11].

An interesting application of the IB problem in communications consists of a source node, an oblivious relay, and a destination node, which is connected to the relay via an error-free link with capacity C. The source node sends codewords over a communication channel and an observation is made at the relay. X and Y are, respectively, the channel input from the source node and output at the relay. The relay is oblivious in the sense that it cannot decode the information message of the source node itself. This feature can be modeled rigorously by assuming that the source and destination nodes make use of a codebook selected at random over a library, while the relay is unaware of such random selection. For example, in a cloud radio access network (C-RAN), each remote radio head (RRH) acts as a relay and is usually constrained to implement only radio functionalities while the baseband functionalities are migrated to the cloud central processor [12]. Considering the relatively simple structure of the RRHs, it is usually prohibitive to let them know the codebooks and random encoding operations, particularly as the network size becomes large. The fact that the relay cannot decode is also supported by secrecy demands, which means that the codebooks known to the source and destination nodes are to be considered absolutely random, as done here.

Due to the oblivious feature, the relaying strategies that require the codebooks to be known at the relay, e.g., decode-and-forward, compute-and-forward, etc. [13,14,15], cannot be applied. Instead, the relay has to perform oblivious processing, i.e., employ strategies in the form of compress-and-forward [16,17,18,19]. In particular, the relay must treat X as a random process with distribution induced by random selection over the codebook library (see [12] and references therein) and has to produce some useful representation Z by simple signal processing and to convey it to the destination node subject to the link constraint C. Then, it makes sense to find Z such that $I(X;Z)$ is maximized.

The IB problem for this kind of communication scenario has been studied in [12,20,21,22,23,24,25,26]. In [20], the IB method was applied to reduce the fronthaul data rate of a C-RAN network. References [21,22], respectively, considered Gaussian scalar and vector channels with IB constraint and investigated the optimal tradeoff between the compression rate and the relevant information. In [23], the bottleneck rate of a frequency-selective scalar Gaussian primitive diamond relay channel was examined. In [24,25], the rate-distortion region of a vector Gaussian system with multiple relays was characterized under the logarithmic loss distortion measure. Reference [12] further extended the work in [25] to a C-RAN network with multiple transmitters and multiple relays and studied the capacity region of this network. However, all of References [12,20,21,22,23,24,25] considered block fading channels and assumed that the perfect channel state information (CSI) was known at both the relay and the destination nodes. In [26], the IB problem of a scalar Rayleigh fading channel was studied. Due to the bottleneck constraint, it was impossible to inform the destination node of the perfect CSI in each channel realization. An upper bound and two achievable schemes were provided in [26] to investigate the bottleneck rate.

In this paper, we extend the work in [26] to the multiple-input multiple-out (MIMO) channel with independent and identically distributed (i.i.d.) Rayleigh fading. This model is relevant for the practical setting of the uplink of a wireless multiuser system where K users send coded uplink signals to a base station. The base station is formed by an RRH with M antennas, connected to a cloud central processor via a digital link of rate C (bottleneck link). The RRH is oblivious to the user codebooks and can apply only simple localized signal processing corresponding to the low-level physical layer functions (i.e., it is an oblivious relay). In current implementations, the RRH quantizes both the uplink pilot symbols and the data-bearing symbols received from the users on each “resource block” (This corresponds roughly to a coherence block of the underlying fading channel in the time-frequency domain) and sends the quantization bits to the cloud processor via the digital link. Here, we simplify the problem, and instead of considering a specific pilot-based channel estimation scheme, we assume that the channel matrix is given perfectly to the relay (remote radiohead), i.e., that the CSI is perfect but local at the relay. Then, we consider an upper bound and specific achievability strategies to maximize the mutual information between the user transmitted signals and the message delivered to the cloud processor, where we allow the relay to operate local oblivious processing as an alternative to direct quantization of both the CSI and the received data-bearing signal.

Intuitively, the relay can split the capacity of the relay-destination link into two parts and convey both the CSI and its observation to the destination node. Hence, in the first and second achievable schemes, the relay transmits the compressed CSI and observation to the destination node. Specifically, in the first scheme, the relay simply compresses the channel matrix as well as its observation and then forwards them to the destination node. Roughly speaking, this is what happens today in “naive” implementation of RRH systems. Therefore, this scheme can be seen as a baseline scheme. However, the capacity allocated for conveying the CSI to the destination in this scheme is proportional to both the channel input dimension and the number of antennas at the relay. To reduce the channel use required for CSI transmission, in the second achievable scheme, the relay first obtains an estimate of the channel input using channel inversion and then transmits the quantized noise levels as well as the compressed noisy signal to the destination node. In contrast to the first scheme, the capacity allocated to CSI transmission in this scheme is only proportional to the channel input dimension.

Due to the explicit CSI transmission through the bottleneck, the performance of the first and second achievable schemes is sensitive to the MIMO channel dimension, especially the channel input dimension. To ensure that it still performs well when the channel dimension grows large, in the third and fourth achievable schemes, the relay does not convey any CSI to the destination node. In the third scheme, the relay first estimates the channel input using channel inversion and then transmits a truncated representation of the estimate to the destination node. In the fourth scheme, the relay first produces the minimum mean-squared error (MMSE) estimate of the channel input and then source-encodes this estimate. Numerical results show that, with simple symbol-by-symbol oblivious relay processing and compression, the lower bounds obtained by the proposed achievable schemes can come close to the upper bound on a wide range of relevant system parameters.

The rest of this paper is organized as follows. In Section 2, a MIMO channel with Rayleigh fading is presented and the IB problem for this system is formulated. Section 3 provides an upper bound to the bottleneck rate. In Section 4, four achievable schemes are proposed, where each scheme satisfies the bottleneck constraint and gives a lower bound to the bottleneck rate. Numerical results are presented in Section 5 before the conclusions in Section 6.

Throughout this paper, we use the following notations. $\mathbb{R}$ and $\mathbb{C}$ denote the real space and the complex space, respectively. Boldface upper (lower) case letters are used to denote matrices (vectors). ${\mathit{I}}_K$ stands for the $K\times K$ dimensional identity matrix and $\mathbf{0}$ denotes the all-zero vector or matrix. Superscript ${(·)}^H$ denotes the conjugated-transpose operation, $\mathbb{E}\left[·\right]$ denotes the expectation operation, and ${[·]}^{+}\triangleq max(·,0)$. ⊗ and ⊙, respectively, denote the Kronecker product and the Hadamard product.

2. Problem Formulation

We consider a system with a source node, an oblivious relay, and a destination node as shown in Figure 1. For convenience, we call the source–relay channel “Channel 1” and the relay–destination channel “Channel 2”. For Channel 1, we consider the following Gaussian MIMO channel with i.i.d. Rayleigh fading:

$$\mathit{y}=\mathit{H}\mathit{x}+\mathit{n},$$

(2)

where $\mathit{x}\in {\mathbb{C}}^{K\times 1}$ and $\mathit{n}\in {\mathbb{C}}^{M\times 1}$ are, respectively, zero-mean circularly symmetric complex Gaussian input and noise with covariance matrices ${\mathit{I}}_K$ and ${\sigma }^2{\mathit{I}}_M$, i.e., $\mathit{x}\sim \mathcal{CN}(\mathbf{0},{\mathit{I}}_K)$ and $\mathit{n}\sim \mathcal{CN}(\mathbf{0},{\sigma }^2{\mathit{I}}_M)$. $\mathit{H}\in {\mathbb{C}}^{M\times K}$ is a random matrix independent of both $\mathit{x}$ and $\mathit{n}$, and the elements of $\mathit{H}$ are i.i.d. zero-mean unit-variance complex Gaussian random variables, i.e., $\mathit{H}\sim \mathcal{CN}(\mathbf{0},{\mathit{I}}_K\otimes {\mathit{I}}_M)$. Let $\rho =\frac{1}{{\sigma }^2}$ denote the signal-to-noise ratio (SNR). Let $\mathit{z}$ denote a useful representation of $\mathit{y}$ produced by the relay for the destination node. $\mathit{x}\to (\mathit{y},\mathit{H})\to \mathit{z}$ thus forms a Markov chain. We assume that the relay node has a direct observation of the channel matrix $\mathit{H}$ while the destination node does not since we consider a Rayleigh fading channel and a capacity-constrained relay–destination link. Then, the IB problem can be formulated as follows:

$$\begin{array}{cc}\hfill \underset{p\left(\mathit{z}\right|\mathit{y},\mathit{H})}{max}& I(\mathit{x};\mathit{z})\hfill \end{array}$$

(3a)

$$\begin{array}{cc}\hfill \mathrm{s}.\mathrm{t}.& I(\mathit{y},\mathit{H};\mathit{z})\le C,\hfill \end{array}$$

(3b)

where C is the bottleneck constraint, i.e., the link capacity of Channel 2. In this paper, we call $I(\mathit{x};\mathit{z})$ the bottleneck rate and $I(\mathit{y},\mathit{H};\mathit{z})$ the compression rate. Obviously, for a joint probability distribution $p(\mathit{x},\mathit{y},\mathit{H})$ determined by (2), problem (3) is a slightly augmented version of IB problem (1). In our problem, we aim to find a conditional distribution $p\left(\mathit{z}\right|\mathit{y},\mathit{H})$ such that bottleneck constraint (3b) is satisfied and the bottleneck rate is maximized, i.e., as much information on $\mathit{x}$ can be extracted from representation $\mathit{z}$.

3. Informed Receiver Upper Bound

As stated in [26], an obvious upper bound to problem (3) can be obtained by letting both the relay and the destination node know the channel matrix $\mathit{H}$. We call the bound in this case the informed receiver upper bound. The IB problem in this case takes on the following form:

$$\begin{array}{cc}\hfill \underset{p\left(\mathit{z}\right|\mathit{y},\mathit{H})}{max}& I(\mathit{x};\mathit{z}|\mathit{H})\hfill \end{array}$$

(4a)

$$\begin{array}{cc}\hfill \mathrm{s}.\mathrm{t}.& I(\mathit{y};\mathit{z}|\mathit{H})\le C.\hfill \end{array}$$

(4b)

In Reference [21], the IB problem for a scalar Gaussian channel with block fading has been studied. In the following theorem, we show that, for the considered MIMO channel with Rayleigh fading, (4) can be decomposed into a set of parallel scalar IB problems and the informed receiver upper bound can be obtained based on the result in [21].

Theorem 1.

For the considered MIMO channel with Rayleigh fading, the informed receiver upper bound, i.e., the optimal objective function of IB problem (4), is

$$R^{\mathrm{ub}}=T{\int }_{\frac{\nu }{\rho }}^{\infty }\left[log\left(1+\rho \lambda \right)-log(1+\nu )\right]f_{\lambda }\left(\lambda \right)d\lambda ,$$

(5)

where $T=min\{K,M\}$, λ is identically distributed as the unordered positive eigenvalues of $\mathit{H}{\mathit{H}}^H$; its probability density function (pdf), i.e., $f_{\lambda }\left(\lambda \right)$, is given in (A17); and ν is chosen such that the following bottleneck constraint is met:

$${\int }_{\frac{\nu }{\rho }}^{\infty }\left(log\frac{\rho \lambda }{\nu }\right)f_{\lambda }\left(\lambda \right)d\lambda =\frac{C}{T}.$$

(6)

Proof. 

See Appendix A. □

Lemma 1.

When $M\to +\infty$ or $\rho \to +\infty$, upper bound $R^{\mathrm{ub}}$ tends asymptotically to C. When $C\to +\infty$, $R^{\mathrm{ub}}$ approaches the capacity of Channel 1, i.e.,

$$\begin{array}{cc}\hfill R^{\mathrm{ub}}& \to I(\mathit{x};\mathit{y},\mathit{H})\hfill \\ \hfill & =T{\int }_0^{\infty }log\left(1+\rho \lambda \right)f_{\lambda }\left(\lambda \right)d\lambda .\hfill \end{array}$$

(7)

Proof. 

See Appendix B. □

4. Achievable Schemes

In this section, we provide four achievable schemes, where each scheme satisfies the bottleneck constraint and gives a lower bound to the bottleneck rate. In the first and second schemes, the relay transmits both its observation and partial CSI to the destination node. In the third and fourth schemes, to avoid transmitting CSI, the relay first estimates $\mathit{x}$ and then sends a representation of the estimate to the destination node.

4.1. Non-Decoding Transmission (NDT) Scheme

Our first achievable scheme assumes that, without decoding $\mathit{x}$, the relay simply source-encodes both $\mathit{y}$ and $\mathit{H}$ and then sends the encoded representations to the destination node. It should be noticed that this scheme is actually reminiscent of the current state-of-the-art in remote antenna head technology, where both the pilot field (corresponding to $\mathit{H}$) and the data field (corresponding to $\mathit{y}$) are quantized and sent to the central processing unit.

Let $\mathit{h}$ denote the vectorization of matrix $\mathit{H}$, and ${\mathit{z}}_1$ and ${\mathit{z}}_2$ denote the representations of $\mathit{h}$ and $\mathit{y}$, respectively. From the definition of $\mathit{H}$ in (2), it is known that $\mathit{h}\sim \mathcal{CN}(\mathbf{0},{\mathit{I}}_{KM})$. Since the elements in $\mathit{h}$ are i.i.d., in the best case, where $I(\mathit{h};{\mathit{z}}_1)$ is minimized for a given total distortion, representation ${\mathit{z}}_1$ introduces the same distortion to each element of $\mathit{h}$. Denote the distortion of each element quantization by D. It can then be readily verified by using ([27], Theorem 10.3.3) that the rate distortion function of source $\mathit{h}$ with total squared-error distortion $KMD$ is given by

$$\begin{array}{cc}\hfill R\left(D\right)& =\underset{f\left({\mathit{z}}_1\right|\mathit{h}):\mathbb{E}\left[d(\mathit{h},{\mathit{z}}_1)\right]\le KMD}{min}I(\mathit{h};{\mathit{z}}_1)\hfill \\ \hfill & =KMlog\frac{1}{D},\hfill \end{array}$$

(8)

where $0<D\le 1$ and $d(\mathit{h},{\mathit{z}}_1)={(\mathit{h}-{\mathit{z}}_1)}^H(\mathit{h}-{\mathit{z}}_1)$ is the squared-error distortion measure. Let ${\mathit{e}}_1$ denote the error vector of quantizing $\mathit{h}$, i.e., ${\mathit{e}}_1=\mathit{h}-{\mathit{z}}_1$. ${\mathit{z}}_1$ and ${\mathit{e}}_1$ are the vectorizations of ${\mathit{Z}}_1$ and ${\mathit{E}}_1$. Hence, $\mathit{H}={\mathit{Z}}_1+{\mathit{E}}_1$. Note that ${\mathit{z}}_1\sim \mathcal{CN}(\mathbf{0},(1-D){\mathit{I}}_{KM})$, ${\mathit{e}}_1\sim \mathcal{CN}(\mathbf{0},D{\mathit{I}}_{KM})$, and ${\mathit{z}}_1$ is independent of ${\mathit{e}}_1$. Hence,

$$\begin{array}{cc}\hfill \mathbb{E}\left[{\mathit{Z}}_1{\mathit{Z}}_1^H\right]& =K(1-D){\mathit{I}}_K,\hfill \\ \hfill \mathbb{E}\left[{\mathit{E}}_1{\mathit{E}}_1^H\right]& =KD{\mathit{I}}_K.\hfill \end{array}$$

(9)

In ([27], Theorem 10.3.3), the achievability of an information rate for a given distortion, e.g., (8), is proven by considering a backward Gaussian test channel. However, the backward Gaussian test channel does not provide an expression of ${\mathit{z}}_1$ or ${\mathit{e}}_1$. Though the specific formulations of ${\mathit{z}}_1$ and ${\mathit{e}}_1$ are not necessary for the analysis in this section, since we are providing an achievable scheme, we still give a feasible ${\mathit{z}}_1$ that satisfies (8) here to make the content more complete. By adding an independent Gaussian noise vector $\mathit{r}\sim \mathcal{CN}(\mathbf{0},\epsilon {\mathit{I}}_{KM})$ with $\epsilon =\frac{D}{1-D}$, to $\mathit{h}$, we get

$$\tilde{\mathit{h}}=\mathit{h}+\mathit{r}.$$

(10)

Obviously, $\tilde{\mathit{h}}\sim \mathcal{CN}\left(\mathbf{0},\frac{1}{1-D}{\mathit{I}}_{KM}\right)$. A representation of $\mathit{h}$ can then be obtained as follows:

$$\begin{array}{cc}\hfill {\mathit{z}}_1& =\frac{1}{1+\epsilon }\tilde{\mathit{h}}\hfill \\ \hfill & =\frac{1}{1+\epsilon }\mathit{h}+\frac{1}{1+\epsilon }\mathit{r}\hfill \\ \hfill & =(1-D)\mathit{h}+(1-D)\mathit{r},\hfill \end{array}$$

(11)

which is actually the MMSE estimate of $\mathit{h}$ obtained from (10). The error vector is then given by

$$\begin{array}{cc}\hfill {\mathit{e}}_1& =\mathit{h}-{\mathit{z}}_1\hfill \\ \hfill & =D\mathit{h}-(1-D)\mathit{r}.\hfill \end{array}$$

(12)

It can be readily verified that ${\mathit{z}}_1$ provided in (11) satisfies (8), ${\mathit{z}}_1\sim \mathcal{CN}(\mathbf{0},(1-D){\mathit{I}}_{KM})$, ${\mathit{e}}_1\sim \mathcal{CN}(\mathbf{0},D{\mathit{I}}_{KM})$, and ${\mathit{z}}_1$ is independent of ${\mathit{e}}_1$.

To meet the bottleneck constraint, we have to ensure that

$$I(\mathit{h},\mathit{y};{\mathit{z}}_1,{\mathit{z}}_2)\le C.$$

(13)

Using the chain rule of mutual information,

$$\begin{array}{cc}\hfill I(\mathit{h},\mathit{y};{\mathit{z}}_1,{\mathit{z}}_2)=& I(\mathit{h},\mathit{y};{\mathit{z}}_1)+I(\mathit{h},\mathit{y};{\mathit{z}}_2|{\mathit{z}}_1)\hfill \\ \hfill =& I(\mathit{h};{\mathit{z}}_1)+I(\mathit{y};{\mathit{z}}_1|\mathit{h})+I(\mathit{y};{\mathit{z}}_2|{\mathit{z}}_1)+I(\mathit{h};{\mathit{z}}_2|{\mathit{z}}_1,\mathit{y}).\hfill \end{array}$$

(14)

Since ${\mathit{z}}_1$ is a representation of $\mathit{h}$, $\mathit{y}$ and ${\mathit{z}}_1$ are conditionally independent given $\mathit{h}$. Similarly, since ${\mathit{z}}_2$ is a representation of $\mathit{y}$, $\mathit{h}$ and ${\mathit{z}}_2$ are conditionally independent given $\mathit{y}$. Hence,

$$\begin{array}{cc}\hfill & I(\mathit{y};{\mathit{z}}_1|\mathit{h})=0,\hfill \\ \hfill & I(\mathit{h};{\mathit{z}}_2|{\mathit{z}}_1,\mathit{y})=0.\hfill \end{array}$$

(15)

From (8), (14), and (15), it is known that, to guarantee constraint (13), $I(\mathit{y};{\mathit{z}}_2|{\mathit{z}}_1)$, which is the information rate at which the relay quantizes $\mathit{y}$ (given ${\mathit{z}}_1$), should satisfy

$$I(\mathit{y};{\mathit{z}}_2|{\mathit{z}}_1)\le C-R\left(D\right).$$

(16)

Obviously, $C-R\left(D\right)>0$ has to be guaranteed, which yields $D>2^{-\frac{C}{KM}}$. Hence, in this section, we always assume $2^{-\frac{C}{KM}}<D\le 1$.

We then evaluate $I(\mathit{y};{\mathit{z}}_2|{\mathit{z}}_1)$. Since $\mathit{H}={\mathit{Z}}_1+{\mathit{E}}_1$, $\mathit{y}$ in (2) can be rewritten as

$$\begin{array}{cc}\hfill \mathit{y}& =\mathit{H}\mathit{x}+\mathit{n}\hfill \\ \hfill & ={\mathit{Z}}_1\mathit{x}+{\mathit{E}}_1\mathit{x}+\mathit{n}.\hfill \end{array}$$

(17)

For a given ${\mathit{Z}}_1$, the second moment of $\mathit{y}$ is $\mathbb{E}\left[\mathit{y}{\mathit{y}}^H|{\mathit{Z}}_1\right]={\mathit{Z}}_1{\mathit{Z}}_1^H+(KD+{\sigma }^2){\mathit{I}}_M$. Denote the eigendecomposition of ${\mathit{Z}}_1{\mathit{Z}}_1^H$ by $\tilde{\mathit{U}}\Omega {\tilde{\mathit{U}}}^H$ and

$$\begin{array}{cc}\hfill \tilde{\mathit{y}}& ={\tilde{\mathit{U}}}^H\mathit{y}\hfill \\ \hfill & ={\tilde{\mathit{U}}}^H{\mathit{Z}}_1\mathit{x}+{\tilde{\mathit{U}}}^H{\mathit{E}}_1\mathit{x}+{\tilde{\mathit{U}}}^H\mathit{n}.\hfill \end{array}$$

(18)

The second moment of $\tilde{\mathit{y}}$ is $\mathbb{E}\left[\tilde{\mathit{y}}{\tilde{\mathit{y}}}^H|{\mathit{Z}}_1\right]=\Omega +(KD+{\sigma }^2){\mathit{I}}_M$. Since ${\mathit{E}}_1$ is unknown, $\tilde{\mathit{y}}$ is not a Gaussian vector. To evaluate $I(\mathit{y};{\mathit{z}}_2|{\mathit{z}}_1)$, we define a new Gaussian vector

$${\mathit{y}}_g={\tilde{\mathit{U}}}^H{\mathit{Z}}_1\mathit{x}+{\mathit{n}}_g,$$

(19)

where ${\mathit{n}}_g\sim \mathcal{CN}(\mathbf{0},(KD+{\sigma }^2){\mathit{I}}_M)$. For a given ${\mathit{Z}}_1$, ${\mathit{y}}_g\sim \mathcal{CN}(\mathbf{0},\Omega +(KD+{\sigma }^2){\mathit{I}}_M)$. The channel in (19) can thus be seen as a set of parallel sub-channels. Let ${\mathit{Z}}_g$ denote a representation of ${\mathit{y}}_g$, and consider the following IB problem:

$$\begin{array}{cc}\hfill \underset{p\left({\mathit{z}}_g\right|{\mathit{y}}_g)}{max}& I(\mathit{x};{\mathit{z}}_g|{\mathit{Z}}_1)\hfill \end{array}$$

(20a)

$$\begin{array}{cc}\hfill \mathrm{s}.\mathrm{t}.& I({\mathit{y}}_g;{\mathit{z}}_g|{\mathit{Z}}_1)\le C-R\left(D\right),\hfill \end{array}$$

(20b)

$$\begin{array}{cc}\hfill & 2^{-\frac{C}{KM}}<D\le 1.\hfill \end{array}$$

(20c)

Obviously, for a given feasible D, problem (20) can be similarly solved as (4) by following the steps in Appendix A. We thus have the following theorem.

Theorem 2.

For a given feasible D, the optimal objective function of IB problem (20) is

$$R^{lb1}=T{\int }_{\frac{\nu }{\gamma }}^{\infty }\left[log\left(1+\gamma \lambda \right)-log(1+\nu )\right]f_{\lambda }\left(\lambda \right)d\lambda ,$$

(21)

where $\gamma =\frac{1-D}{KD+{\sigma }^2}$; the pdf of λ, i.e., $f_{\lambda }\left(\lambda \right)$, is given by (A17); and ν is chosen such that the following bottleneck constraint is met:

$${\int }_{\frac{\nu }{\gamma }}^{\infty }\left(log\frac{\gamma \lambda }{\nu }\right)f_{\lambda }\left(\lambda \right)d\lambda =\frac{C-R\left(D\right)}{T}.$$

(22)

Proof. 

See Appendix C. □

Since for a given ${\mathit{Z}}_1$, (19) can be seen as a set of parallel scalar Gaussian sub-channels, according to ([21], (16)), the representation of ${\mathit{y}}_g$, i.e., ${\mathit{z}}_g$, can be constructed by adding independent fading and Gaussian noise to each element of ${\mathit{y}}_g$. Denote

$$\begin{array}{cc}\hfill {\mathit{z}}_g& =\Psi {\mathit{y}}_g+{\mathit{n}}_g^{\prime }\hfill \\ \hfill & =\Psi {\tilde{\mathit{U}}}^H{\mathit{z}}_1\mathit{x}+\Psi {\mathit{n}}_g+{\mathit{n}}_g^{\prime },\hfill \end{array}$$

(23)

where $\Psi$ is a diagonal matrix with nonnegative and real diagonal entries, and ${\mathit{n}}_g^{\prime }\sim \mathcal{CN}\left(\mathbf{0},{\mathit{I}}_M\right)$. Note that ${\mathit{y}}_g$ in (19) and its representation ${\mathit{z}}_g$ in (23) are only auxiliary variables. What we are really interested in is the representation of $\mathit{y}$ and the corresponding bottleneck rate. Hence, we also add fading $\Psi$ and Gaussian noise ${\mathit{n}}_g^{\prime }$ to $\tilde{\mathit{y}}$ in (18) and obtain the following representation:

$$\begin{array}{cc}\hfill {\mathit{z}}_2& =\Psi \tilde{\mathit{y}}+{\mathit{n}}_g^{\prime }\hfill \\ \hfill & =\Psi {\tilde{\mathit{U}}}^H{\mathit{Z}}_1\mathit{x}+\Psi {\tilde{\mathit{U}}}^H{\mathit{E}}_1\mathit{x}+\Psi {\tilde{\mathit{U}}}^H\mathit{n}+{\mathit{n}}_g^{\prime }.\hfill \end{array}$$

(24)

In the following lemma, we show that, by transmitting representations ${\mathit{z}}_1$ and ${\mathit{z}}_2$ to the destination node, $R^{lb1}$ is an achievable lower bound to the bottleneck rate and the bottleneck constraint is satisfied.

Lemma 2.

If the representation of $\mathit{h}$, i.e., ${\mathit{z}}_1$ resulting from (8), is forwarded to the destination node for each channel realization, with observations $\mathit{y}$ and ${\mathit{y}}_g$ in (17) and (18) and representations ${\mathit{z}}_2$ and ${\mathit{z}}_g$ in (24) and (23), we have

$$\begin{array}{cc}\hfill I(\mathit{y};{\mathit{z}}_2|{\mathit{Z}}_1)& \le I({\mathit{y}}_g;{\mathit{z}}_g|{\mathit{Z}}_1),\hfill \end{array}$$

(25)

$$\begin{array}{cc}\hfill I(\mathit{x};{\mathit{z}}_2|{\mathit{Z}}_1)& \ge I(\mathit{x};{\mathit{z}}_g|{\mathit{Z}}_1),\hfill \end{array}$$

(26)

where (25) indicates that $I(\mathit{y};{\mathit{z}}_2|{\mathit{Z}}_1)\le C-R\left(D\right)$ and (26) gives $I(\mathit{x};{\mathit{z}}_2|{\mathit{Z}}_1)\ge R^{lb1}$.

Proof. 

See Appendix D. □

Lemma 2 shows that, by representing $\mathit{h}$ and $\tilde{\mathit{y}}$ using ${\mathit{z}}_1$ and ${\mathit{z}}_2$ in (11) and (24), respectively, lower bound $R^{lb1}$ is achievable and the bottleneck constraint is satisfied.

Lemma 3.

When $M\to +\infty$,

$$R^{lb1}\to T\left[log\left(1+\gamma M\right)-log\left(1+\gamma M{2}^{-\frac{C-R\left(D\right)}{T}}\right)\right].$$

(27)

When $\rho \to +\infty$, $R^{lb1}$ tends to a constant, which can be obtained by letting $\gamma =\frac{1-D}{KD}$ and using (21). In addition, when $C\to +\infty$, there exists a small D such that $R^{lb1}$ approaches the capacity of Channel 1, i.e.,

$$\begin{array}{cc}\hfill R^{lb1}& \to I(\mathit{x};\mathit{y},\mathit{H})\hfill \\ \hfill & =T{\int }_0^{\infty }log\left(1+\rho \lambda \right)f_{\lambda }\left(\lambda \right)d\lambda .\hfill \end{array}$$

(28)

Proof. 

See Appendix E. □

Remark 1.

Denote the limit in (27) by $R_0^{lb1}=T\left[log\left(1+\gamma M\right)-log\left(1+\gamma M{2}^{-\frac{C-R\left(D\right)}{T}}\right)\right]$ for convenience. It can be readily verified that $0\le R_0^{lb1}\le C$. From (8), it is known that $R\left(D\right)$ is also a function of M. Moreover, as stated after (16), we always assume $2^{-\frac{C}{KM}}<D\le 1$ in this section such that $C-R\left(D\right)>0$. Hence, when $M\to +\infty$, D approaches 1 and γ tends to 0. All this makes it difficult to obtain further concise expression of $R_0^{lb1}$. We investigate the effect of M on $R^{lb1}$ in Section 5 by simulation.

4.2. Quantized Channel Inversion (QCI) Scheme When $K\le M$

In our second scheme, the relay first obtains an estimate of the channel input using channel inversion and then transmits the quantized noise levels as well as the compressed noisy signal to the destination node.

In particular, we apply the pseudo inverse matrix of $\mathit{H}$, i.e., ${\left({\mathit{H}}^H\mathit{H}\right)}^{-1}{\mathit{H}}^H$, to $\mathit{y}$ and obtain the zero-forcing estimate of $\mathit{x}$ as follows:

$$\begin{array}{cc}\hfill \tilde{\mathit{x}}& ={\left({\mathit{H}}^H\mathit{H}\right)}^{-1}{\mathit{H}}^H\mathit{y}\hfill \\ \hfill & =\mathit{x}+{\left({\mathit{H}}^H\mathit{H}\right)}^{-1}{\mathit{H}}^H\mathit{n}\hfill \\ \hfill & \triangleq \mathit{x}+\tilde{\mathit{n}}.\hfill \end{array}$$

(29)

For a given channel matrix $\mathit{H}$, $\tilde{\mathit{n}}\sim \mathcal{CN}(\mathbf{0},\mathit{A})$, where $\mathit{A}={\sigma }^2{\left({\mathit{H}}^H\mathit{H}\right)}^{-1}$. Let $\mathit{A}={\mathit{A}}_1+{\mathit{A}}_2$, where ${\mathit{A}}_1$ and ${\mathit{A}}_2$, respectively, consist of the diagonal and off-diagonal elements of $\mathit{A}$, i.e., ${\mathit{A}}_1=\mathit{A}\odot {\mathit{I}}_K$ and ${\mathit{A}}_2=\mathit{A}-{\mathit{A}}_1$. If $\mathit{H}$ could be perfectly transmitted to the destination node, the bottleneck rate could be obtained by following similar steps in Appendix A. However, since $\mathit{H}$ follows a non-degenerate continuous distribution and the bottleneck constraint is finite, as shown in the previous subsection, this is not possible. To reduce the number of bits per channel use required for informing the destination node of the channel information, we only convey a compressed version of ${\mathit{A}}_1$ and consider a set of independent scalar Gaussian sub-channels.

Specifically, we force each diagonal entry of ${\mathit{A}}_1$ to belong to a finite set of quantized levels by adding artificial noise, i.e., by introducing physical degradation. We fix a finite grid of J positive quantization points $\mathcal{B}=\{b_1,\cdots ,b_J\}$, where $b_1\le b_2\le \cdots \le b_{J-1}<b_J$, $b_J=+\infty$, and define the following ceiling operation:

$${\lceil a\rceil }_{\mathcal{B}}=arg\underset{b\in \mathcal{B}}{min}\{a\le b\}.$$

(30)

Then, by adding a Gaussian noise vector ${\tilde{\mathit{n}}}^{\prime }\sim \mathcal{CN} \left(\mathbf{0}, \right.$$\left.\mathrm{diag}\left\{{\lceil a_1\rceil }_{\mathcal{B}}-a_1,\cdots ,{\lceil a_K\rceil }_{\mathcal{B}}-a_K\right\}\right)$, which is independent of everything else, to (29), a degraded version of $\tilde{\mathit{x}}$ can be obtained as follows:

$$\begin{array}{cc}\hfill \widehat{\mathit{x}}& =\tilde{\mathit{x}}+{\tilde{\mathit{n}}}^{\prime }\hfill \\ \hfill & =\mathit{x}+\tilde{\mathit{n}}+{\tilde{\mathit{n}}}^{\prime }\hfill \\ \hfill & \triangleq \mathit{x}+\widehat{\mathit{n}},\hfill \end{array}$$

(31)

where $\widehat{\mathit{n}}\sim \mathcal{CN}\left(\mathbf{0},{\mathit{A}}_1^{\prime }+{\mathit{A}}_2\right)$ for a given $\mathit{H}$ and ${\mathit{A}}_1^{\prime }\triangleq \mathrm{diag}\left\{{\lceil a_1\rceil }_{\mathcal{B}},\cdots ,{\lceil a_K\rceil }_{\mathcal{B}}\right\}$. Obviously, due to ${\mathit{A}}_2$, the elements in noise vector $\widehat{\mathit{n}}$ are correlated.

To evaluate the bottleneck rate, we consider a new variable

$${\widehat{\mathit{x}}}_g=\mathit{x}+{\widehat{\mathit{n}}}_g,$$

(32)

where ${\widehat{\mathit{n}}}_g\sim \mathcal{CN}\left(\mathbf{0},{\mathit{A}}_1^{\prime }\right)$. Obviously, (32) can be seen as K parallel scalar Gaussian sub-channels with noise power ${\lceil a_k\rceil }_{\mathcal{B}}$ for each sub-channel. Since each quantized noise level ${\lceil a_k\rceil }_{\mathcal{B}}$ only has J possible values, it is possible for the relay to inform the destination node of the channel information via the constrained link. Note that, from the definition of $\mathit{A}$ in (29), it is known that $a_k,\forall k\in \mathcal{K}\triangleq \{1,\cdots ,K\}$ are correlated. The quantized noise levels ${\lceil a_k\rceil }_{\mathcal{B}},\forall k\in \mathcal{K}$ are thus also correlated. Hence, we can jointly source-encode ${\lceil a_k\rceil }_{\mathcal{B}},\forall k\in \mathcal{K}$ to further reduce the number of bits used for CSI transmission. For convenience, we define a space $\Xi =\left\{(j_1,\cdots ,j_K)|\forall j_k\in \mathcal{J},k\in \mathcal{K}\right\}$, where $\mathcal{J}=\{1,\cdots ,J\}$. It is obvious that there are a total of $J^K$ points in this space. Let $\xi =(j_1,\cdots ,j_K)$ denote a point in space $\Xi$ and define the following probability mass function (pmf):

$$P_{\xi }=\mathrm{Pr}\left\{{\lceil a_1\rceil }_{\mathcal{B}}=b_{j_1},\cdots ,{\lceil a_K\rceil }_{\mathcal{B}}=b_{j_K}\right\}.$$

(33)

The joint entropy of ${\lceil a_k\rceil }_{\mathcal{B}},\forall k\in \mathcal{K}$, i.e., the number of bits used for jointly source-encoding ${\lceil a_k\rceil }_{\mathcal{B}},\forall k\in \mathcal{K}$, is thus given by

$$\begin{array}{cc}\hfill H_{\mathrm{joint}}& =\sum _{\xi \in \Xi }-P_{\xi }log{P}_{\xi }.\hfill \end{array}$$

(34)

Then, the IB problem for (32) takes on the following form:

$$\begin{array}{cc}\hfill \underset{p\left({\widehat{\mathit{z}}}_g\right|{\widehat{\mathit{x}}}_g)}{max}& I(\mathit{x};{\widehat{\mathit{z}}}_g|{\mathit{A}}_1^{\prime })\hfill \end{array}$$

(35a)

$$\begin{array}{cc}\hfill \mathrm{s}.\mathrm{t}.& I({\widehat{\mathit{x}}}_g;{\widehat{\mathit{z}}}_g|{\mathit{A}}_1^{\prime })\le C-H_{\mathrm{joint}},\hfill \end{array}$$

(35b)

where ${\widehat{\mathit{z}}}_g$ is a representation of ${\widehat{\mathit{x}}}_g$.

Note that, as stated above, there are a total of $J^K$ points in space $\Xi$. The pmf $P_{\xi }$ thus has $J^K$ possible values, and it becomes difficult to obtain the joint entropy $H_{\mathrm{joint}}$ from (34) (even numerically) when J or K is large. To reduce the computational complexity, we consider the (slightly) suboptimal but far more practical entropy coding of each noise level ${\lceil a_k\rceil }_{\mathcal{B}}$ separately and obtain the following sum of individual entropies:

$$H_{\mathrm{sum}}=\sum _{k=1}^K{H}_k,$$

(36)

where $H_k$ denotes the entropy of ${\lceil a_k\rceil }_{\mathcal{B}}$ or the number of bits used for informing the destination node of noise level ${\lceil a_k\rceil }_{\mathcal{B}}$. In Appendix F, we show that $a_k,\forall k\in \mathcal{K}$ are marginally identically inverse chi squared distributed with $M-K+1$ degrees of freedom and that their pdf is given in (A44). Hence,

$$\begin{array}{cc}\hfill H_{\mathrm{sum}}& =K{H}_0\hfill \\ \hfill & =-K\sum _{j=1}^J{P}_{j}log{P}_j,\hfill \end{array}$$

(37)

where $P_j=\mathrm{Pr}\left\{{\lceil a\rceil }_{\mathcal{B}}=b_j\right\}$ can be obtained from (A45) and a follows the same distribution as $a_k$. Since $P_j$ only has J possible values, the computational complexity of calculating $H_{\mathrm{sum}}$ is proportional to J. Using the chain rule of entropy and the fact that conditioning reduces entropy, we know that $H_{\mathrm{joint}}\le H_{\mathrm{sum}}$. In Section 5, the gap between $H_{\mathrm{joint}}$ and $H_{\mathrm{sum}}$ is investigated by simulation. Replacing $H_{\mathrm{joint}}$ in (35b) with $H_{\mathrm{sum}}$, we get the following: IB problem

$$\begin{array}{cc}\hfill \underset{p\left({\widehat{\mathit{z}}}_g\right|{\widehat{\mathit{x}}}_g)}{max}& I(\mathit{x};{\widehat{\mathit{z}}}_g|{\mathit{A}}_1^{\prime })\hfill \end{array}$$

(38a)

$$\begin{array}{cc}\hfill \mathrm{s}.\mathrm{t}.& I({\widehat{\mathit{x}}}_g;{\widehat{\mathit{z}}}_g|{\mathit{A}}_1^{\prime })\le C-K{H}_0.\hfill \end{array}$$

(38b)

The optimal solution of this problem is given in the following theorem.

Theorem 3.

If ${\mathit{A}}_1^{\prime }$ is conveyed to the destination node for each channel realization, the optimal objective function of IB problem (38) is

$$R^{lb2}=\sum _{j=1}^{J-1}K{P}_j\left[log\left(1+{\rho }_j\right)-log(1+{\rho }_j{2}^{-c_j})\right].$$

(39)

where ${\rho }_j=\frac{1}{b_j}$, $c_j={\left[log\frac{{\rho }_j}{\nu }\right]}^{+}$, and ν is chosen such that the following bottleneck constraint is met:

$$\sum _{j=1}^{J-1}K{P}_j{c}_j=C-K{H}_0.$$

(40)

Proof. 

See Appendix F. □

Since (32) can be seen as K parallel scalar Gaussian sub-channels, according to ([21], (16)), the representation of ${\widehat{\mathit{x}}}_g$, i.e., ${\widehat{\mathit{z}}}_g$, can be constructed by adding independent fading and Gaussian noise to each element of ${\widehat{\mathit{x}}}_g$. Denote

$$\begin{array}{cc}\hfill {\widehat{\mathit{z}}}_g& =\Phi {\widehat{\mathit{x}}}_g+{\widehat{\mathit{n}}}_g^{\prime }\hfill \\ \hfill & =\Phi \mathit{x}+\Phi {\widehat{\mathit{n}}}_g+{\widehat{\mathit{n}}}_g^{\prime },\hfill \end{array}$$

(41)

where $\Phi$ is a diagonal matrix with positive and real diagonal entries, and ${\widehat{\mathit{n}}}_g^{\prime }\sim \mathcal{CN}\left(\mathbf{0},{\mathit{I}}_K\right)$. Note that, similar to ${\mathit{y}}_g$ and ${\mathit{z}}_g$ in the previous subsection, ${\widehat{\mathit{x}}}_g$ in (32) and its representation ${\widehat{\mathit{z}}}_g$ in (41) are also auxiliary variables. What we are really interested in is the representation of $\widehat{\mathit{x}}$ and the corresponding bottleneck rate. Hence, we also add fading $\Phi$ and Gaussian noise ${\widehat{\mathit{n}}}_g^{\prime }$ to $\widehat{\mathit{x}}$ in (31) and obtain its representation as follows:

$$\begin{array}{cc}\hfill \mathit{z}& =\Phi \widehat{\mathit{x}}+{\widehat{\mathit{n}}}_g^{\prime }\hfill \\ \hfill & =\Phi \mathit{x}+\Phi \widehat{\mathit{n}}+{\widehat{\mathit{n}}}_g^{\prime }.\hfill \end{array}$$

(42)

In the following lemma, we show that, by transmitting quantized noise levels ${\lceil a_k\rceil }_{\mathcal{B}},\forall k\in \mathcal{K}$ and representation $\mathit{z}$ to the destination node, $R^{lb2}$ is an achievable lower bound to the bottleneck rate and the bottleneck constraint is satisfied.

Lemma 4.

If ${\mathit{A}}_1^{\prime }$ is forwarded to the destination node for each channel realization, with signal vectors $\widehat{\mathit{x}}$ and ${\widehat{\mathit{x}}}_g$ in (31) and (32), and their representations $\mathit{z}$ and ${\widehat{\mathit{z}}}_g$ in (42) and (41), we have

$$\begin{array}{cc}\hfill I(\widehat{\mathit{x}};\mathit{z}|{\mathit{A}}_1^{\prime })& \le I({\widehat{\mathit{x}}}_g;{\widehat{\mathit{z}}}_g|{\mathit{A}}_1^{\prime }),\hfill \end{array}$$

(43)

$$\begin{array}{cc}\hfill I(\mathit{x};\mathit{z}|{\mathit{A}}_1^{\prime })& \ge I(\mathit{x};{\widehat{\mathit{z}}}_g|{\mathit{A}}_1^{\prime }),\hfill \end{array}$$

(44)

where (43) indicates that $I(\widehat{\mathit{x}};\mathit{z}|{\mathit{A}}_1^{\prime })\le C-K{H}_0$ and (44) gives $I(\mathit{x};\mathit{z}|{\mathit{A}}_1^{\prime })\ge R^{lb1}$.

Proof. 

See Appendix G. □

Lemma 5.

When $M\to +\infty$ or $\rho \to +\infty$, we can always find a sequence of quantization points $\mathcal{B}=\{b_1,\cdots ,b_J\}$ such that $R^{lb2}\to C$. When $C\to +\infty$,

$$\begin{array}{cc}\hfill R^{lb2}& \to K\mathbb{E}\left[log\left(1+\frac{1}{a}\right)\right]\hfill \\ \hfill & \le I(\mathit{x};\mathit{y},\mathit{H}),\hfill \end{array}$$

(45)

where the expectation can be calculated by using the pdf of a in (A44) and $I(\mathit{x};\mathit{y},\mathit{H})$ is the capacity of Channel 1.

Proof. 

See Appendix H. □

For the sake of simplicity, we may choose the quantization levels as quantiles such that we obtain the uniform pmf $P_j=\frac{1}{J}$. The lower bound (39) can thus be simplified as

$$R^{lb2}=\sum _{j=1}^{J-1}\frac{K}{J}\left[log\left(1+{\rho }_j\right)-log(1+{\rho }_j{2}^{-c_j})\right],$$

(46)

and the bottleneck constraint (40) becomes

$$\sum _{j=1}^{J-1}{\left[log\frac{{\rho }_j}{\nu }\right]}^{+}=\frac{JC}{K}-JB,$$

(47)

where $B=logJ$ can be seen as the number of bits required for quantizing each diagonal entry of ${\mathit{A}}_1$. Since ${\rho }_1\ge \cdots \ge {\rho }_{J-1}$, from the strict convexity of the problem, we know that there must exist a unique integer $1\le l\le J-1$ such that [28]

$$\begin{array}{cc}\hfill & \sum _{j=1}^{l}log\frac{{\rho }_j}{\nu }=\frac{JC}{K}-JB,\hfill \\ \hfill & {\rho }_j\le \nu ,\forall l+1\le j\le J-1.\hfill \end{array}$$

(48)

Hence, $\nu$ can be obtained from

$$log\nu =\sum _{j=1}^l\frac{log{\rho }_j}{l}-\frac{JC}{lK}+\frac{JB}{l},$$

(49)

and $R^{lb1}$ can be calculated as follows:

$$R^{lb2}=\sum _{j=1}^l\frac{K}{J}\left[log\left(1+{\rho }_j\right)-log(1+\nu )\right].$$

(50)

Then, we only need to test the above condition for $l=1,2,3,\cdots$ until (48) is satisfied. Note that, to ensure $R^{lb2}>0$, $\frac{JC}{K}-JB$ in (47) has to be positive, i.e., $B<\frac{C}{K}$. Moreover, though choosing the quantization levels as quantiles makes it easier to calculate $R^{lb2}$, the results in Lemma 5 may not hold in this case since the choice of quantization points $\mathcal{B}=\{b_1,\cdots ,b_J\}$ is restricted.

4.3. Truncated Channel Inversion (TCI) Scheme When $K\le M$

Both the NDT and QCI schemes proposed in the preceding two subsections require that the relay transmits partial CSI to the destination node. Specifically, in the NDT scheme, channel matrix $\mathit{H}$ is compressed and conveyed to the destination node. Hence, the channel use required for transmitting compressed $\mathit{H}$ is proportional to K and M. In contrast, the number of bits required for transmitting quantized noise levels in the QCI scheme is proportional to K and B. Due to the bottleneck constraint, the performances of the NDT and QCI schemes are thus sensitive to the MIMO channel dimension, especially K. To ensure that it still performs well when the channel dimension is large, in this subsection, the relay first estimates $\mathit{x}$ using channel inversion and then transmits a truncated representation of the estimate to the destination node.

In particular, as in the previous subsection, we first obtain the zero-forcing estimate of $\mathit{x}$ using channel inversion, i.e.,

$$\begin{array}{cc}\hfill \tilde{\mathit{x}}& ={\left({\mathit{H}}^H\mathit{H}\right)}^{-1}{\mathit{H}}^H\mathit{y}\hfill \\ \hfill & =\mathit{x}+{\left({\mathit{H}}^H\mathit{H}\right)}^{-1}{\mathit{H}}^H\mathit{n}.\hfill \end{array}$$

(51)

As given in Appendix A, the unordered eigenvalues of ${\mathit{H}}^H\mathit{H}$ are ${\lambda }_k,\forall k\in \mathcal{K}$. Let ${\lambda }_{min}=min\{{\lambda }_k,\forall k\in \mathcal{K}\}$. Note that, though the interfering terms can be nulled out by a zero-forcing equalizer, the noise may be greatly amplified when the channel is noisy. Therefore, we put a threshold ${\lambda }_{\mathrm{th}}$ on ${\lambda }_{min}$ such that zero capacity is allocated for states with ${\lambda }_{min}<{\lambda }_{\mathrm{th}}$.

Specifically, when ${\lambda }_{min}<{\lambda }_{\mathrm{th}}$, the relay does not transmit the observation, while when ${\lambda }_{min}\ge {\lambda }_{\mathrm{th}}$, the relay takes $\tilde{\mathit{x}}$ as the new observation and transmits a compressed version of $\tilde{\mathit{x}}$ to the destination node. The information about whether to transmit the observation is encoded into a $0-1$ sequence and is also sent to the destination node. Then, we need to solve the source coding problem at the relay, i.e., encoding blocks of $\tilde{\mathit{x}}$ when ${\lambda }_{min}\ge {\lambda }_{\mathrm{th}}$. For convenience, we use $\Delta$ to denote event “${\lambda }_{min}\ge {\lambda }_{\mathrm{th}}$”. Here, we choose $p\left(\mathit{z}\right|\tilde{\mathit{x}},\Delta )$ to be a conditional Gaussian distribution:

$$\mathit{z}=\left\{\begin{array}{cc}\tilde{\mathit{x}}+\mathit{q},\hfill & \mathrm{if}\Delta \hfill \\ \varnothing ,\hfill & \mathrm{otherwise}\hfill \end{array}\right.,$$

(52)

where $\mathit{q}\sim \mathcal{CN}(\mathbf{0},D{\mathit{I}}_K)$ is independent of the other variables. It can be easily found from (52) that $I(\mathit{x};\mathit{z}|{\lambda }_{min}<{\lambda }_{\mathrm{th}})=0$ and $I(\tilde{\mathit{x}};\mathit{z}|{\lambda }_{min}<{\lambda }_{\mathrm{th}})=0$. Hence, we consider the following modified IB problem:

$$\begin{array}{cc}\hfill \underset{D}{max}& P_{\mathrm{th}}I(\mathit{x};\mathit{z}|\Delta )\hfill \end{array}$$

(53a)

$$\begin{array}{cc}\hfill \mathrm{s}.\mathrm{t}.& P_{\mathrm{th}}I(\tilde{\mathit{x}};\mathit{z}|\Delta )\le C-H_{\mathrm{th}},\hfill \end{array}$$

(53b)

where $P_{\mathrm{th}}=\mathrm{Pr}\left\{\Delta \right\}$ and $H_{\mathrm{th}}$ is a binary entropy function with parameter $P_{\mathrm{th}}$.

Since we assume $K\le M$ in this subsection, as stated in Appendix A, ${\mathit{H}}^H\mathit{H}\sim {\mathcal{CW}}_K(M,{\mathit{I}}_K)$. Then, according to ([29], Proposition 2.6) and ([29], Proposition 4.7), $P_{\mathrm{th}}$ is given by

$$P_{\mathrm{th}}=\frac{det\mathit{\psi }}{{\prod }_{k=1}^K(M-k)!{\prod }_{k=1}^K(K-k)!},$$

(54)

where

$$\begin{array}{cc}\hfill & \mathit{\psi }=\left[\begin{array}{ccc}{\psi }_0& \cdots & {\psi }_{K-1}\\ ⋮& \ddots & ⋮\\ {\psi }_{K-1}& \cdots & {\psi }_{2K-2}\end{array}\right]=\left[\left({\psi }_{i+j-2}\right)\right],\hfill \\ \hfill & {\psi }_{i+j-2}={\int }_{{\lambda }_{\mathrm{th}}}^{\infty }{\mu }^{M-K+i+j-2}{e}^{-\mu }d\mu .\hfill \end{array}$$

(55)

When $K=M$, using ([30], Theorem 3.2), a more concise expression of $P_{\mathrm{th}}$ can be obtained as follows:

$$\begin{array}{cc}\hfill P_{\mathrm{th}}& ={\int }_{2{\lambda }_{\mathrm{th}}}^{\infty }\frac{K}{2}{e}^{-\mu K/2}d\mu \hfill \\ \hfill & =e^{-{\lambda }_{\mathrm{th}}K}.\hfill \end{array}$$

(56)

Note that, in (56), the lower bound of the integral is $2{\lambda }_{\mathrm{th}}$ rather than ${\lambda }_{\mathrm{th}}$. This is because, in this paper, the elements of $\mathit{H}$ are assumed to be i.i.d. zero-mean unit-variance complex Gaussian random variables, while in [30], the real and imaginary parts of the elements in $\mathit{H}$ are independent standard normal variables.

Given condition $\Delta$, let ${\tilde{\mathit{x}}}_g$ denote a zero-mean circularly symmetric complex Gaussian random vector with the same second moment as $\tilde{\mathit{x}}$, i.e., ${\tilde{\mathit{x}}}_g\sim \mathcal{CN}\left(\mathbf{0},\mathbb{E}\left[\tilde{\mathit{x}}{\tilde{\mathit{x}}}^H|\Delta \right]\right)$, and ${\tilde{\mathit{z}}}_g={\tilde{\mathit{x}}}_g+\mathit{q}$. $P_{\mathrm{th}}I({\tilde{\mathit{x}}}_g;{\tilde{\mathit{z}}}_g|\Delta )$ is then achievable if $P_{\mathrm{th}}I({\tilde{\mathit{x}}}_g;{\tilde{\mathit{z}}}_g|\Delta )\le C-H_{\mathrm{th}}$. Hence, let

$$\begin{array}{cc}\hfill P_{\mathrm{th}}I({\tilde{\mathit{x}}}_g;{\tilde{\mathit{z}}}_g|\Delta )& =P_{\mathrm{th}}logdet\left({\mathit{I}}_K+\frac{1}{D}\mathbb{E}\left[\tilde{\mathit{x}}{\tilde{\mathit{x}}}^H|\Delta \right]\right)\hfill \\ \hfill & =C-H_{\mathrm{th}}.\hfill \end{array}$$

(57)

To calculate D from (57), we denote the eigendecomposition of ${\mathit{H}}^H\mathit{H}$ by $\mathit{V}\tilde{\Lambda }{\mathit{V}}^H$, where $\mathit{V}$ is a unitary matrix in which the columns are the eigenvectors of ${\mathit{H}}^H\mathit{H}$, $\tilde{\Lambda }$ is a diagonal matrix in which the diagonal elements are unordered eigenvalues ${\lambda }_k,\forall k\in \mathcal{K}$, and $\mathit{V}$ and $\tilde{\Lambda }$ are independent. Then, from (51),

$$\begin{array}{cc}\hfill \mathbb{E}\left[\tilde{\mathit{x}}{\tilde{\mathit{x}}}^H|\Delta \right]& ={\mathit{I}}_K+{\sigma }^2\mathbb{E}\left[{\left({\mathit{H}}^H\mathit{H}\right)}^{-1}|\Delta \right],\hfill \\ \hfill & ={\mathit{I}}_K+{\sigma }^2\mathbb{E}\left[\mathit{V}{\tilde{\Lambda }}^{-1}{\mathit{V}}^H|\Delta \right],\hfill \\ \hfill & ={\mathit{I}}_K+{\sigma }^2\mathbb{E}\left[\frac{1}{\lambda }|\Delta \right]{\mathit{I}}_K.\hfill \end{array}$$

(58)

Based on [31], the joint pdf of the unordered eigenvalues ${\lambda }_k,\forall k\in \mathcal{K}$ under condition $\Delta$ is given by

$$f({\lambda }_1,\cdots ,{\lambda }_K|\Delta )=\frac{1}{P_{\mathrm{th}}K!}\prod _{i=1}^K\frac{e^{-{\lambda }_i}{\lambda }_i^{M-K}}{(K-i)!(M-i)!}\prod _{i<j}^K{({\lambda }_i-{\lambda }_j)}^2.$$

(59)

The marginal pdf of one of the eigenvalues can thus be obtained by integrating out all the other eigenvalues. Taking ${\lambda }_1$ for example, we have

$$f_{{\lambda }_1}\left({\lambda }_1\right|\Delta )={\int }_{{\lambda }_{\mathrm{th}}}^{\infty }\cdots {\int }_{{\lambda }_{\mathrm{th}}}^{\infty }f({\lambda }_1,\cdots ,{\lambda }_K|\Delta )d{\lambda }_2\cdots d{\lambda }_K.$$

(60)

Then,

$$\begin{array}{cc}\hfill \mathbb{E}\left[\frac{1}{\lambda }|\Delta \right]& =\mathbb{E}\left[\frac{1}{{\lambda }_1}|\Delta \right]\hfill \\ \hfill & ={\int }_{{\lambda }_{\mathrm{th}}}^{\infty }\frac{1}{{\lambda }_1}{f}_{{\lambda }_1}\left({\lambda }_1\right|\Delta )d{\lambda }_1.\hfill \end{array}$$

(61)

Combining (57), (58), and (61), D can be calculated as follows:

$$D=\frac{1+{\sigma }^2\mathbb{E}\left[\frac{1}{\lambda }|\Delta \right]}{2^{\frac{C-H_{\mathrm{th}}}{P_{\mathrm{th}}K}}-1}.$$

(62)

Remark 2.

Note that we show in Appendix I that, when $K=M$ and ${\lambda }_{\mathrm{th}}=0$, the integral in (61) diverges. $\mathbb{E}\left[\frac{1}{\lambda }|\Delta \right]$ thus does not exist in this case. Therefore, without special instructions, the results derived in this subsection are for the cases with $K=M$ and ${\lambda }_{\mathrm{th}}>0$ or with $K<M$ and ${\lambda }_{\mathrm{th}}\ge 0$.

With (57), rate $P_{\mathrm{th}}I({\tilde{\mathit{x}}}_g;{\tilde{\mathit{z}}}_g|\Delta )$ is achievable. Due to the fact that Gaussian input maximizes the mutual information of a Gaussian additive noise channel, we have $I(\tilde{\mathit{x}};\mathit{z}|\Delta )\le I({\tilde{\mathit{x}}}_g;{\tilde{\mathit{z}}}_g|\Delta )$. $P_{\mathrm{th}}I(\tilde{\mathit{x}};\mathit{z}|\Delta )$ is thus also achievable.

The next step is to evaluate the resulting achievable bottleneck rate, i.e., $I(\mathit{x};\mathit{z})$. To this end, we first obtain the following lower bound to $I(\mathit{x};\mathit{z}|\Delta )$ from the fact that conditioning reduces differential entropy,

$$\begin{array}{cc}\hfill I(\mathit{x};\mathit{z}|\Delta )=& h\left(\mathit{z}\right|\Delta )-h(\mathit{z}|\mathit{x},\Delta )\hfill \\ \hfill \ge & h\left(\mathit{z}\right|\mathit{H},\Delta )-h(\mathit{z}|\mathit{x},\Delta ).\hfill \end{array}$$

(63)

Then, we evaluate the differential entropies $h\left(\mathit{z}\right|\mathit{H},\Delta )$ and $h\left(\mathit{z}\right|\mathit{x},\Delta )$. From (51) and (52), it is known that $\mathit{z}$ is conditionally Gaussian given $\mathit{H}$ and $\Delta$. Hence,

$$\begin{array}{cc}\hfill h\left(\mathit{z}\right|\mathit{H},\Delta )=& \mathbb{E}\left[log{\left(\pi e\right)}^{K}det\left({\mathit{I}}_K+{\sigma }^2{\left({\mathit{H}}^H\mathit{H}\right)}^{-1}+D{\mathit{I}}_K\right)|\Delta \right]\hfill \\ \hfill =& \mathbb{E}\left[log{\left(\pi e\right)}^{K}det\left({\mathit{I}}_K+{\sigma }^2{\tilde{\Lambda }}^{-1}+D{\mathit{I}}_K\right)|\Delta \right]\hfill \\ \hfill =& K\mathbb{E}\left[log\left(\pi e\right)\left(1+D+\frac{{\sigma }^2}{\lambda }\right)|\Delta \right].\hfill \end{array}$$

(64)

On the other hand, using the fact that Gaussian distribution maximizes the entropy over all distributions with the same variance ([27], Theorem 8.6.5), we have

$$\begin{array}{cc}\hfill h\left(\mathit{z}\right|\mathit{x},\Delta )=& h(\mathit{z}-\mathit{x}|\Delta )\hfill \\ \hfill =& h({\left({\mathit{H}}^H\mathit{H}\right)}^{-1}{\mathit{H}}^H\mathit{n}+\mathit{q}|\Delta )\hfill \\ \hfill \le & log{\left(\pi e\right)}^{K}det\left({\sigma }^2\mathbb{E}\left[{\left({\mathit{H}}^H\mathit{H}\right)}^{-1}|\Delta \right]+D{\mathit{I}}_K\right)\hfill \\ \hfill =& Klog\left(\pi e\right)\left(D+{\sigma }^2\mathbb{E}\left[\frac{1}{\lambda }|\Delta \right]\right).\hfill \end{array}$$

(65)

Substituting (64) and (65) into (63), we can obtain a lower bound to $I(\mathit{x};\mathit{z})$, as shown in the following theorem.

Theorem 4.

When $K\le M$, with truncated channel inversion, a lower bound to $I(\mathit{x};\mathit{z})$ can be obtained as follows:

$$R^{lb3}=P_{\mathrm{th}}K\mathbb{E}\left[log\left(1+D+\frac{{\sigma }^2}{\lambda }\right)|\Delta \right]-P_{\mathrm{th}}Klog\left(D+{\sigma }^2\mathbb{E}\left[\frac{1}{\lambda }|\Delta \right]\right),$$

(66)

where $P_{\mathrm{th}}$ and D are, respectively, given in (54) and (62), and the expectations can be calculated by using pdf (60).

Lemma 6.

Using Jensen’s inequality on convex function $log(1+1/x)$ and concave function $logx$, we can get a lower bound to $R^{lb3}$, i.e.,

$${\check{R}}^{lb3}=P_{\mathrm{th}}Klog\left(1+D+\frac{{\sigma }^2}{\mathbb{E}\left[\lambda |\Delta \right]}\right)-P_{\mathrm{th}}Klog\left(D+{\sigma }^2\mathbb{E}\left[\frac{1}{\lambda }|\Delta \right]\right),$$

(67)

and an upper bound to $R^{lb3}$, i.e.,

$${\widehat{R}}^{lb3}=P_{\mathrm{th}}Klog\left(1+D+{\sigma }^2\mathbb{E}\left[\frac{1}{\lambda }|\Delta \right]\right)-P_{\mathrm{th}}Klog\left(D+{\sigma }^2\mathbb{E}\left[\frac{1}{\lambda }|\Delta \right]\right).$$

(68)

Remark 3.

Obviously, ${\check{R}}^{lb3}$ is also a lower bound to $I(\mathit{x};\mathit{z})$. For ${\widehat{R}}^{lb3}$, it is not an upper bound to $I(\mathit{x};\mathit{z})$ since it is derived after lower bound $R^{lb3}$. However, we can assess how good the lower bounds $R^{lb3}$ and ${\check{R}}^{lb3}$ are by comparing them with ${\widehat{R}}^{lb3}$.

Lemma 7.

When $M\to +\infty$, $R^{lb3}$, ${\check{R}}^{lb3}$, and ${\widehat{R}}^{lb3}$ all tend asymptotically to C. When $\rho \to +\infty$, $R^{lb3}$, ${\check{R}}^{lb3}$, and ${\widehat{R}}^{lb3}$ all tend asymptotically to $C-H_{\mathrm{th}}$. In addition, when $C\to +\infty$, $R^{lb3}$, ${\check{R}}^{lb3}$, and ${\widehat{R}}^{lb3}$ all approach constants, which can be respectively obtained by setting $D=0$ in (66)–(68).

Proof. 

See Appendix J. □

When $K<M$ and ${\lambda }_{\mathrm{th}}=0$, it is obvious that $P_{\mathrm{th}}=1$, $H_{\mathrm{th}}=0$, and $\mathbb{E}\left[\lambda \right]=M$. Since ${\mathit{H}}^H\mathit{H}\sim {\mathcal{CW}}_K(M,{\mathit{I}}_K)$, ${\left({\mathit{H}}^H\mathit{H}\right)}^{-1}$ follows a complex inverse Wishart distribution. Hence, $\mathbb{E}\left[\frac{1}{\lambda }\right]=\frac{1}{M-K}$. Then, from Theorem 4 and Lemma 6, we have the following lemma.

Lemma 8.

When $K<M$ and ${\lambda }_{\mathrm{th}}=0$,

$$R^{lb3}=K\mathbb{E}\left[log\left(1+D+\frac{{\sigma }^2}{\lambda }\right)\right]-Klog\left(D+\frac{{\sigma }^2}{M-K}\right),$$

(69)

$${\check{R}}^{lb3}=Klog\left(1+D+\frac{{\sigma }^2}{M}\right)-Klog\left(D+\frac{{\sigma }^2}{M-K}\right),$$

(70)

and

$${\widehat{R}}^{lb3}=Klog\left(1+D+\frac{{\sigma }^2}{M-K}\right)-Klog\left(D+\frac{{\sigma }^2}{M-K}\right),$$

(71)

where

$$D=\frac{1+\frac{{\sigma }^2}{M-K}}{2^{\frac{C}{K}}-1}.$$

(72)

Remark 4.

When $K<M$, ${\lambda }_{\mathrm{th}}=0$, and $\frac{{\sigma }^2}{M-K}$ is small (e.g., when ρ is large, i.e., ${\sigma }^2$ is small, or when $M-K$ is large), ${\widehat{R}}^{lb3}-{\check{R}}^{lb3}\approx 0$. In this case, ${\check{R}}^{lb3}$ is close to ${\widehat{R}}^{lb3}$ and is thus also close to $R^{lb3}$. Then, we can use ${\check{R}}^{lb3}$ instead of $R^{lb3}$ to lower bound $I(\mathit{x};\mathit{z})$ since it has a more concise expression.

4.4. MMSE Estimate at the Relay

In this subsection, we assume that the relay first produces the MMSE estimate of $\mathit{x}$ given $(\mathit{y},\mathit{H})$ and then source-encodes this estimate.

Denote

$$\mathit{F}={\left(\mathit{H}{\mathit{H}}^H+{\sigma }^2{\mathit{I}}_M\right)}^{-1}\mathit{H}.$$

(73)

The MMSE estimate of $\mathit{x}$ is thus given by

$$\begin{array}{cc}\hfill \overline{\mathit{x}}& ={\mathit{F}}^H\mathit{y}\hfill \\ \hfill & ={\mathit{F}}^H\mathit{H}\mathit{x}+{\mathit{F}}^H\mathit{n}.\hfill \end{array}$$

(74)

Then, we consider the following modified IB problem:

$$\begin{array}{cc}\hfill \underset{p\left(\mathit{z}\right|\overline{\mathit{x}})}{max}& I(\mathit{x};\mathit{z})\hfill \end{array}$$

(75a)

$$\begin{array}{cc}\hfill \mathrm{s}.\mathrm{t}.& I(\overline{\mathit{x}};\mathit{z})\le C.\hfill \end{array}$$

(75b)

Note that, since matrix $\mathit{H}{\mathit{H}}^H+{\sigma }^2{\mathit{I}}_K$ in (73) is always invertible, the results obtained in this subsection always hold no matter $K\le M$ or $K>M$.

Analogous to the previous subsection, we define

$$\begin{array}{cc}\hfill & \mathit{z}=\overline{\mathit{x}}+\mathit{q},\hfill \\ \hfill & {\overline{\mathit{x}}}_g\sim \mathcal{CN}\left(\mathbf{0},\mathbb{E}\left[\overline{\mathit{x}}{\overline{\mathit{x}}}^H\right]\right),\hfill \\ \hfill & {\overline{\mathit{z}}}_g={\overline{\mathit{x}}}_g+\mathit{q},\hfill \end{array}$$

(76)

where $\mathit{q}$ has the same definition as in (52), and

$$\mathbb{E}\left[\overline{\mathit{x}}{\overline{\mathit{x}}}^H\right]=\mathbb{E}\left[{\mathit{F}}^H\mathit{H}{\mathit{H}}^H\mathit{F}+{\sigma }^2{\mathit{F}}^H\mathit{F}\right].$$

(77)

Let

$$\begin{array}{cc}\hfill I({\overline{\mathit{x}}}_g;{\overline{\mathit{z}}}_g)& =logdet\left({\mathit{I}}_K+\frac{\mathbb{E}\left[\overline{\mathit{x}}{\overline{\mathit{x}}}^H\right]}{D}\right)\hfill \\ \hfill & =C.\hfill \end{array}$$

(78)

Then, rate $I({\overline{\mathit{x}}}_g;{\overline{\mathit{z}}}_g)$ is achievable and D can be calculated from (78). Since $I(\overline{\mathit{x}};\mathit{z})\le I({\overline{\mathit{x}}}_g;{\overline{\mathit{z}}}_g)$, $I(\overline{\mathit{x}};\mathit{z})$ is thus also achievable.

In the following, we obtain a lower bound to $I(\mathit{x};\mathit{z})$ by evaluating $h\left(\mathit{z}\right|\mathit{H})$ and $h\left(\mathit{z}\right|\mathit{x})$ separately and then by using

$$\begin{array}{cc}\hfill I(\mathit{x};\mathit{z})=& h\left(\mathit{z}\right)-h\left(\mathit{z}\right|\mathit{x})\hfill \\ \hfill \ge & h\left(\mathit{z}\right|\mathit{H})-h(\mathit{z}\left|\mathit{x}\right).\hfill \end{array}$$

(79)

First, since $\mathit{z}$ is conditionally Gaussian given $\mathit{H}$, we have

$$h\left(\mathit{z}\right|\mathit{H})=\mathbb{E}\left[log{\left(\pi e\right)}^{K}det\left({\mathit{F}}^H\mathit{H}{\mathit{H}}^H\mathit{F}+{\sigma }^2{\mathit{F}}^H\mathit{F}+D{\mathit{I}}_K\right)\right].$$

(80)

Next, based on the fact that conditioning reduces differential entropy and Gaussian distribution maximizes the entropy over all distributions with the same variance [32], we have

$$\begin{array}{cc}\hfill h\left(\mathit{z}\right|\mathit{x})& =h\left(\mathit{z}-\mathbb{E}\left(\mathit{z}\right|\mathit{x}\left)\right|\mathit{x}\right)\hfill \\ \hfill & =h\left(\left({\mathit{F}}^H\mathit{H}-\mathbb{E}\left[{\mathit{F}}^H\mathit{H}\right]\right)\mathit{x}+{\mathit{F}}^H\mathit{n}+\mathit{q}|\mathit{x}\right)\hfill \\ \hfill & \le h\left(\left({\mathit{F}}^H\mathit{H}-\mathbb{E}\left[{\mathit{F}}^H\mathit{H}\right]\right)\mathit{x}+{\mathit{F}}^H\mathit{n}+\mathit{q}\right)\hfill \\ \hfill & \le log{\left(\pi e\right)}^{K}det\left(\mathit{G}\right),\hfill \end{array}$$

(81)

where

$$\begin{array}{cc}\hfill \mathit{G}& =\mathbb{E}\left[\left({\mathit{F}}^H\mathit{H}-\mathbb{E}\left[{\mathit{F}}^H\mathit{H}\right]\right)\left({\mathit{H}}^H\mathit{F}-\mathbb{E}\left[{\mathit{H}}^H\mathit{F}\right]\right)+{\sigma }^2{\mathit{F}}^H\mathit{F}\right]+D{\mathit{I}}_K\hfill \\ \hfill & =\mathbb{E}\left[{\mathit{F}}^H\mathit{H}{\mathit{H}}^H\mathit{F}\right]-\mathbb{E}\left[{\mathit{F}}^H\mathit{H}\right]\mathbb{E}\left[{\mathit{H}}^H\mathit{F}\right]+{\sigma }^2\mathbb{E}\left[{\mathit{F}}^H\mathit{F}\right]+D{\mathit{I}}_K.\hfill \end{array}$$

(82)

Combining (79)–(81), we can get a lower bound to $I(\mathit{x};\mathit{z})$ as shown in the following theorem.

Theorem 5.

With the MMSE estimate at the relay, a lower bound to $I(\mathit{x};\mathit{z})$ can be obtained as follows:

$$\begin{array}{cc}\hfill R^{lb4}& =T\mathbb{E}\left[log\left(\frac{\lambda }{\lambda +{\sigma }^2}+D\right)\right]+(K-T)logD\hfill \\ \hfill & -Klog\left\{\frac{T}{K}\mathbb{E}\left[\frac{\lambda }{\lambda +{\sigma }^2}\right]-\frac{T^2}{K^2}{\left(\mathbb{E}\left[\frac{\lambda }{\lambda +{\sigma }^2}\right]\right)}^2+D\right\},\hfill \end{array}$$

(83)

where

$$D=\frac{\frac{T}{K}\mathbb{E}\left[\frac{\lambda }{\lambda +{\sigma }^2}\right]}{2^{\frac{C}{K}}-1},$$

(84)

and the expectations can be calculated by using the pdf of λ in (A17).

Proof. 

See Appendix K. □

Lemma 9.

When $M\to +\infty$ or when $K\le M$ and $\rho \to +\infty$, lower bound $R^{lb4}$ tends asymptotically to C. When $K\le M$ and $C\to +\infty$,

$$R^{lb4}\to K\mathbb{E}\left[log\left(\frac{\lambda }{\lambda +{\sigma }^2}\right)\right]-Klog\left\{\mathbb{E}\left[\frac{\lambda }{\lambda +{\sigma }^2}\right]-{\left(\mathbb{E}\left[\frac{\lambda }{\lambda +{\sigma }^2}\right]\right)}^2\right\}.$$

(85)

Proof. 

See Appendix L. □

5. Numerical Results

In this section, we evaluate the lower bounds obtained by different achievable schemes proposed in Section 4 and compare them with the upper bound derived in Section 3. Before showing the numerical results, we first give the following lemma, which compares the bottleneck rate of the NDT scheme with those of the other three schemes in the $C\to +\infty$ case.

Lemma 10.

When $C\to +\infty$, the NDT scheme outperforms the other three schemes, i.e.,

$$R^{lb1}\ge max\left\{R^{lb2},R^{lb3},R^{lb4}\right\}.$$

(86)

Proof. 

See Appendix M. □

Remark 5.

Besides the proof in Appendix M, we can also explain Lemma 10 from a more intuitive perspective. When $C\to +\infty$, the destination node can obtain perfect $\mathit{y}$ and $\mathit{H}$ from the relay by using the NDT scheme. The bottleneck rate is thus determined by the capacity of Channel 1. In the QCI scheme, though the destination node can obtain perfect signal vector and noise power of each channel, the correlation between the elements of the noise vector is neglected since the off-diagonal entries of $\mathit{A}$ are not considered. The bottleneck rate obtained by the QCI scheme is thus upper bounded by the capacity of Channel 1. As for the TCI or MMSE schemes, the destination node can obtain perfect $\tilde{\mathit{x}}$ or $\overline{\mathit{x}}$ from the relay. However, the bottleneck rate in these two cases is not only affected by the capacity of Channel 1 but is also limited by the performance of zero-forcing or MMSE estimation since the estimation inevitably incurs a loss of information. Hence, the NDT scheme has a better performance when $C\to +\infty$.

In the following, we give the numerical results. Note that, when performing the QCI scheme, we choose the quantization levels as quantiles for the sake of convenience.

Figure 2 depicts $R^{lb1}$ versus distortion D under different configurations of SNR $\rho$. It can be found from this figure that $R^{lb1}$ first increases and then decreases with D. It is thus important to find a good D to maximize $R^{lb1}$. Since it is difficult to obtain the explicit expression of (21), it is not easy to strictly analyze the relationship between $R^{lb1}$ and D. However, we can intuitively explain Figure 2 as follows. When using the NDT scheme, the relay quantizes both $\mathit{h}$ and $\mathit{y}$. Due to the bottleneck constraint C, there exists a tradeoff. When D is small, the estimation error of $\mathit{h}$ is small. The destination node can get more CSI, and $R^{lb1}$ thus increases with D. When D grows large, though more capacity in C is allocated for quantizing $\mathit{y}$, the estimation error of $\mathit{h}$ is large. Hence, $R^{lb1}$ decreases with D. In the following simulation process, when implementing the NDT scheme, we vary D, calculate $R^{lb1}$ using (21), and then let $R^{lb1}$ be the maximum value.

In Figure 3 and Figure 4, we performed Monte Carlo simulations to obtain joint entropy $H_{\mathrm{joint}}$ in (34) and the sum of individual entropies $H_{\mathrm{sum}}$ in (37). Note that, as stated in Section 4.2, the complexities of calculating $H_{\mathrm{joint}}$ and $H_{\mathrm{sum}}$ are, respectively, proportional to $J^K$ and J. Hence, when J or K is large, it becomes quite difficult to obtain $H_{\mathrm{joint}}$. For example, when $B=4$ and $K=4$, we have $J=16$ and $J^K=$ 65,536, i.e., there are 65,536 points in space $\Xi$. To obtain a reliable pmf $P_{\xi }$ for each point, the number of channel realizations has to be much greater than 65,536.

Figure 3 shows that the gap between $H_{\mathrm{joint}}$ and $H_{\mathrm{sum}}$ is small. In addition, as M increases, $H_{\mathrm{joint}}$ approaches $H_{\mathrm{sum}}$ quickly, indicating that the dependence between ${\lceil a_k\rceil }_{\mathcal{B}},\forall k\in \mathcal{K}$ becomes weak. This can be explained by considering an extreme case where $M\to +\infty$. Based on the definition of $\mathit{H}$ and the strong law of large numbers, we almost surely have ${\mathit{H}}^H\mathit{H}-M{\mathit{I}}_K\to \mathbf{0}$ when $M\to +\infty$. Hence, $\mathit{A}-\frac{{\sigma }^2}{M}{\mathit{I}}_K\to \mathbf{0}$. ${\lceil a_k\rceil }_{\mathcal{B}},\forall k\in \mathcal{K}$ are thus almost independent.

When $M=K$ and K increases, Figure 4 shows that there exists an obvious increase in the gap between $H_{\mathrm{joint}}$ and $H_{\mathrm{sum}}$. Hence, when $M=K$ and K increases, the correlation between ${\lceil a_k\rceil }_{\mathcal{B}},\forall k\in \mathcal{K}$ is enhanced. We will thus obtain a gain to $R^{lb2}$ if we use $H_{\mathrm{joint}}$ instead of $H_{\mathrm{sum}}$. However, we would like to point out the following: First, it can be found from Figure 4 that, when $M>K$, this trend becomes less evident. Second, as shown in the following results, when $K\ge 4$, since the QCI scheme uses a lot of capacity in C to quantize ${\lceil a_k\rceil }_{\mathcal{B}},\forall k\in \mathcal{K}$, its performance is not as good as the TCI scheme or MMSE scheme. Third, when K or B is large, it becomes difficult to obtain $H_{\mathrm{joint}}$. Therefore, when implementing the QCI scheme in the following, we obtain $R^{lb2}$ by using $H_{\mathrm{sum}}$, i.e., quantizing ${\lceil a_k\rceil }_{\mathcal{B}},\forall k\in \mathcal{K}$ separately.

In Figure 5 and Figure 6, we investigate the effect of threshold ${\lambda }_{\mathrm{th}}$ on $R^{lb3}$ for the cases with $K=M$ and $K<M$, respectively. From these two figures, several observations can be made. First, when $K=M$, and $\rho$ or K is small, $R^{lb3}$ increases greatly and then decreases with ${\lambda }_{\mathrm{th}}$, indicating that the choice of ${\lambda }_{\mathrm{th}}$ has a significant impact on $R^{lb3}$. It is thus important to look for a good ${\lambda }_{\mathrm{th}}$ to maximize $R^{lb3}$ in these cases. Second, when $K=M$, and K as well as $\rho$ are large or when $K<M$, $R^{lb3}$ first remains unchanged and then monotonically decreases with $R^{lb3}$. In these cases, a small ${\lambda }_{\mathrm{th}}$ is good enough to guarantee a large $R^{lb3}$ and a search for ${\lambda }_{\mathrm{th}}$ can thus be avoided. For example, when $K<M$, we can set ${\lambda }_{\mathrm{th}}=0$, based on which a simpler expression of $R^{lb3}$ is given in (69). For the case with $K=M$, since $\mathbb{E}\left[\frac{1}{\lambda }\right]$ does not exist when ${\lambda }_{\mathrm{th}}=0$, we can set ${\lambda }_{\mathrm{th}}$ to be a fixed small number.

In Figure 7 and Figure 8, we compare $R^{lb3}$ with its upper bound ${\widehat{R}}^{lb3}$ and lower bound ${\check{R}}^{lb3}$. As expected, $R^{lb3}$, ${\widehat{R}}^{lb3}$, and ${\check{R}}^{lb3}$ all increase with M and $\rho$. When M or $\rho$ is small, there is a small gap between $R^{lb3}$ and ${\widehat{R}}^{lb3}$, and a small gap between $R^{lb3}$ and ${\check{R}}^{lb3}$. As M and $\rho$ increase, these gaps narrow rapidly and the curves almost coincide, which verifies Remark 2. As a result, when $M-K$ or $\rho$ is large, we can set ${\lambda }_{\mathrm{th}}=0$ and use ${\check{R}}^{lb3}$ in (70) to lower bound $I(\mathit{x};\mathit{z})$ since it has a more concise expression.

In Figure 9 and Figure 10, the upper bound $R^{\mathrm{ub}}$ and lower bounds obtained by different schemes are depicted versus SNR $\rho$. Several observations can be made from these two figures. First, as expected, all bounds increase with $\rho$. Second, when K, M, and $\rho$ are small, the NDT scheme outperforms the other achievable schemes. However, as these parameters increase, the performance of the NDT scheme deteriorates rapidly. This is because, when K, M, and $\rho$ are small, the performance of the considered system is mainly limited by the capacity of Channel 1, and the NDT scheme works well since the destination node can extract more information from the compressed observation of the relay and CSI. However, when K and M increase, the NDT scheme requires too many channel uses for CSI transmission. Third, the QCI scheme can obtain a good performance when K is small. Of course, as stated at the beginning of Section 4.3, the number of bits required for transmitting quantized noise levels in the QCI scheme is proportional to K and B. Hence, the performance of the QCI scheme varies significantly when K and B change. Moreover, it is also shown that the performance of the TCI scheme is worse than that of the MMSE scheme in the low SNR regime while getting quite close to that of the MMSE scheme in the high SNR regime. When $\rho$ grows large, the lower bounds obtained by the TCI and MMSE schemes both approach C and are larger than those obtained by the NDT and QCI schemes.

In Figure 11 and Figure 12, the effect of the bottleneck constraint C is investigated. From Figure 11, it can be found that, as C increases, all bounds grow and converge to different constants, which can be calculated based on Lemmas 1, 3, 5, 7 and 9. Figure 11 also shows that, thanks to CSI transmission, the NDT and QCI schemes outperform the TCI and MMSE schemes when C is large. By comparing these two figures, it can be found that, in Figure 11, no bound approaches C, even for the case with $C=20$, while in Figure 12, it is possible for $R^{\mathrm{ub}}$, $R^{lb3}$, and $R^{lb4}$ to approach C. For example, when $K=M=4$ and $C\le 30$, $R^{\mathrm{ub}},R^{lb3},R^{lb4}\to C$. This is because the bottleneck rate is limited by the capacity of Channel 1 and C. In Figure 11, since K and M are small, the capacity of Channel 1 is smaller than C. Hence, the bounds of course will not approach C. In Figure 12, more multi-antenna gains can be obtained due to larger K and M. The capacity of Channel 1 is thus larger than C in some cases (e.g., $K=M=4$ and $C\le 30$). Hence, $R^{\mathrm{ub}}$, $R^{lb3}$, and $R^{lb4}$ may approach C in these cases. Note that, as shown in Figure 11, since $B<\frac{C}{K}$ is not satisfied, $R^{lb4}=0$ when $C\le 30$.

In Figure 13 and Figure 14, the effect of M is investigated for different configurations of $\rho$. These two figures show that $R^{\mathrm{ub}}$, $R^{lb2}$, $R^{lb3}$, and $R^{lb4}$ all increase monotonically with M and that, as M grows, $R^{lb3}$ as well as $R^{lb4}$ become very close to $R^{\mathrm{ub}}$. For $R^{lb1}$, except the $M=3$ case in Figure 13, $R^{lb1}$ monotonically decreases with M since the relay has to transmit more channel information to the destination node.

In Figure 15 and Figure 16, we set $K=M$ and depict the upper and lower bounds versus K or M. In Figure 15, we fix C to 50, while in Figure 16, we set $C=8K$, which makes sense since the bottleneck constraint should scale with the number of degrees of freedom of the input signal $\mathit{x}$. Since we choose the quantization levels as quantiles when performing the QCI scheme, as stated at the end of Section 4.2, $B<\frac{C}{K}$ should be satisfied. Hence, in Figure 15 and Figure 16, we only consider $B=1,2,4$ bits when performing the QCI scheme. When $K=M$ and they grow simultaneously, the capacity of Channel 1 increases due to the muti-antenna gains. Hence, for a fixed C, Figure 15 shows that all bounds increase first. When K or M grows large, $R^{lb3}$ and $R^{lb4}$ approach the bottleneck constraint C while $R^{lb2}$ decreases for all values of B. This is because the number of bits per channel use required for informing the destination node of ${\mathit{A}}_1^{\prime }$ in the QCI scheme is proportional to K while CSI transmission is unnecessary for the TCI and MMSE schemes. For the NDT scheme, since the number of bits required for quantizing $\mathit{H}$ is proportional to both K and M, there is only an increase when K grows from 1 to 2. After that, $R^{lb1}$ decreases monotonically and has the worst performance. In contrast, when $C=8K$, the bottleneck rate of the system is mainly limited by C. Hence, Figure 16 shows that all bounds except $R^{lb1}$ increase almost linearly with K and that $R^{\mathrm{ub}}$, $R^{lb3}$, and $R^{lb4}$ are quite close to C.

6. Conclusions

This work extends the IB problem of the scalar case in [26] to the case of MIMO Rayleigh fading channels. Due to the information bottleneck constraint, the destination node cannot obtain a perfect CSI from the relay. Hence, we provide an upper bound to the bottleneck rate by assuming that the destination node can obtain a perfect CSI at no cost. Moreover, we also provide four achievable schemes, where each scheme satisfies the bottleneck constraint and gives a lower bound to the bottleneck rate. Our results show that, with simple symbol-by-symbol relay processing and compression, we can obtain a bottleneck rate close to the upper bound on a wide range of relevant system parameters. Note that tightening the upper bound is a challenge for future studies. In addition, although we have focused on a MIMO channel with one relay, we plan to extend the problem to considering the case of multiple parallel relays, which is particularly relevant to the centralized processing of multiple remote antennas, as in the so-called C-RAN architectures.