Transmit Precoding via Block Diagonalization with Approximately Optimized Distance Measures for Limited Feedback in Dense Cellular Networks with Multiantenna Base Stations

Abstract

This study introduces distance metrics for quantized-channel-based precoding in multiuser multiantenna systems, aiming to enhance spectral efficiency in dense cellular networks. Traditional metrics, such as the chordal distance, face limitations when dealing with scenarios involving limited feedback and multiple receive antennas. We address these challenges by developing distance measures that more accurately reflect network conditions, including the impact of intercell interference. Our distance measures are specifically designed to approximate the instantaneous rate of each user by estimating the unknown components during the quantization stage. This approach enables the associated users to efficiently estimate their achievable rates during the quantization process. Our distance measures are specifically designed for block diagonalization precoding, a method known for its computational efficiency and strong performance in multi-user multiple-input and multiple-output systems. The proposed metrics outperform conventional distance measures, particularly in environments where feedback resources are constrained, as is often the case in 5G and emerging 6G networks. The enhancements are especially significant in dense cellular networks, where accurate channel state information is critical for maintaining high spectral efficiency. Our findings suggest that these new distance measures offer a robust solution for improving the performance of limited-feedback-based precoding in cellular networks.

1. Introduction

The multiple-input and multiple-output (MIMO) system has been widely studied over several decades as a key factor in enhancing the spectral efficiency of wireless communication systems. It is well established that employing an MIMO architecture can lead to a linear increase in spectral efficiency, where the slope of this increase is determined by the minimum number of transmit and receive antennas [1,2]. In point-to-point MIMO systems, appropriate joint signal processing among the received signals ensures this linear increase, even in the absence of transmit precoding [3]. However, increasing the number of antennas in user equipment can be challenging in practical applications, as user devices typically need to be portable, and users often require extended operating time or battery life.

To overcome this issue, multiuser MIMO (MU-MIMO) systems have been explored in downlink channels, where multiple users collectively form a cooperative MIMO system, thereby enabling a linear increase in spectral efficiency as a function of the number of transmit antennas, provided that the total number of users’ antennas exceeds the number of transmit antennas. In MU-MIMO channels, capacity is achieved through a nonlinear transmission technique known as dirty-paper coding [4]. However, due to practical challenges such as computational complexity, simpler linear precoding methods have also been extensively studied as near-optimal transmission strategies [5,6]. In this context, this study focuses on a linear precoding method known as block diagonalization.

In MU-MIMO channels, the level of knowledge regarding the channel state information (CSI) at the transmitter (CSIT) is critical to the system’s performance. Particularly in frequency division duplex (FDD) mode, the transmitter cannot directly track the downlink channel, requiring the receiver to estimate and feed back the CSI to the transmitter to maintain a certain level of spectral efficiency. A well-known method for this is limited feedback, where each user estimates, quantizes, and feeds back its channel direction information (CDI) to the transmitter based on a predefined codebook. With limited feedback, it has been demonstrated that the number of bits used for quantization must increase as a function of the signal-to-noise ratio (SNR) to maintain a constant performance gap compared to assuming perfect CSIT [7,8,9]. Additionally, when the number of receive antennas exceeds one, the choice of distance measure for quantization becomes more critical, as the CDI is represented by a unitary matrix derived from the channel matrix [10]. Although these studies offer valuable insights into the relationship between the number of feedback bits and the corresponding performance, they typically assume a single transmitter (or access point), meaning the analytical results do not account for intercell interference in practical cellular networks. As a result, some key observations from these studies may not apply in real-world cellular environments. For instance, the requirement to increase the number of feedback bits as a function of SNR to maintain a constant gap from the perfect CSIT scenario may not hold if there is persistent intercell interference. In addition, an appropriate distance measure for linear precoding must be carefully designed to account for cellular interference, especially when each BS operates in a dense cellular network.

Thus, recent studies have focused on modeling realistic cellular networks using stochastic geometry [11,12,13,14,15,16,17,18,19]. Stochastic geometry is employed to model the locations of network elements such as base stations (BSs) and users. This approach allows for the calculation of average system performance and facilitates analysis that can predict expected outcomes even when the number of BSs in the network is very large. Consequently, it has been widely applied to various wireless communication scenarios, including MIMO in homogeneous cellular networks [13,14], heterogeneous networks [15], MIMO combined with intelligent reflecting surfaces [16], massive MIMO systems in cellular networks [17], cell-free massive MIMO systems [18], and multicell systems using nonorthogonal multiple access [19].

In particular, by applying limited feedback models to downlink transmissions from randomly distributed BSs, the impact of limited feedback has been analyzed from different perspectives. For instance, the authors in [20] derived a lower bound on the optimal number of feedback bits, which was then used to investigate the growth rate of this optimal number as a function of the channel coherence time. Zero-forcing beamforming was used in [20] as a linear beamforming method, assuming that each user has a single receive antenna, and the optimization problem was formulated to maximize net spectral efficiency. Similarly, Ref. [21] examined the optimal number of feedback bits that maximizes the net ergodic secrecy rate. The authors in [22] extended the findings of [20] to scenarios where each user has multiple receive antennas, and the corresponding optimal number was investigated. The authors in [23] further extended the model to a MIMO heterogeneous scenario and derived a closed-form expression for the average rate.

Specifically, when $N_r=1$, in limited feedback systems where each user quantizes CDI for precoding [7], a typical approach for CDI quantization is to minimize the distance between the channel direction vector and a codeword. This distance can be calculated solely based on the channel direction vector and the codeword, with chordal distance being a common metric. It is known to achieve near-optimal performance, as the system’s SINR can be approximated as a decreasing function of the chordal distance. However, when $N_r>1$, chordal distance is not an optimal distance measure for maximizing the system’s spectral efficiency. This is because chordal distance only minimizes the trace of the quantization error matrix between the channel direction matrix and a codeword, ignoring other factors influencing performance. The system’s achievable rate is an increasing function of the determinant of a matrix that includes the quantization error matrix. In this case, as shown in the literature [10], using a performance-metric-based distance measure can yield better results than measures based solely on the channel direction matrix and codeword, such as chordal distance. However, previous studies on distance measures for $N_r>1$ have primarily assumed network scenarios where multicell interference is not significant. Thus, there is a lack of studies designing appropriate distance measures that account for both multiuser interference from limited feedback and multicell interference from densely distributed base stations in cellular networks. The main goal of our study is to address this research gap by designing an appropriate distance measure for $N_r>1$, approximating the achievable spectral efficiency of each user.

Therefore, in this study, we focus on designing an appropriate distance measure for limited-feedback-based precoding, which is used when each user quantizes its CDI for limited feedback. We consider block diagonalization (BD) as the linear precoding method, given that it has been shown to be a highly efficient strategy in terms of both performance and computational cost. Specifically, we summarize the main contributions of this study as follows:

• We first propose distance measures that estimate the approximated spectral efficiency achievable by the target user. Since each user must apply the distance measure before actual downlink transmission begins, they cannot know the exact value of the achievable rate. Therefore, the components of achievable rate that are unknown to the user at the quantization stage are replaced with their expected values.
• We propose two types of close approximations for distance measure and compare the performance of the proposed measures with the typical distance metrics commonly used for measuring distances between matrices. As our proposed distance measures account for the BS topology and small-scale channel gains, they achieve significantly higher performance compared to the conventional distance measures.

The remaining part of this paper is organized as follows: Section 2 explains the system model, and Section 3 includes the process of designing the proposed distance measures. Section 4 presents simulation results that investigate the performance of the proposed distance metrics. At last, Section 5 provides the conclusion of this paper.

Notations: Matrices are represented by uppercase bold letters, and column vectors are denoted by lowercase bold letters. For a matrix $\mathbf{A}$, $\mathrm{tr}\left(\mathbf{A}\right)$, ${\left(\mathbf{A}\right)}^H$, and ${\left[\mathbf{A}\right]}_{i,j}$ denote the trace, the complex conjugate transpose, and the $(i,j)$-th element of $\mathbf{A}$, respectively. The symbol $\parallel ·\parallel$ denotes the vector norm, and $|·|$ represents the absolute value of a complex number. The function $\Gamma (·)$ stands for the Gamma function, $\mathrm{Pr}(·)$ indicates the probability of an event, and $\mathbb{E}(·)$ represents the expectation. The matrix ${\mathbf{I}}_m$ is an $m\times m$ identity matrix. The sets $\mathbb{N}$, $\mathbb{R}$, and $\mathbb{C}$ represent the sets of natural numbers, real numbers, and complex numbers, respectively.

2. System Model and Preliminaries

2.1. Network Model

We consider a cellular network in which the locations of BSs and users are distributed following two independent homogeneous Poisson point processes (PPPs). Each BS is equipped with $N_t$ transmit antennas, and each user has $N_r$ receive antennas. The density of the PPP modeling the BS locations is denoted by $\mu$, and the location vector of BS i is denoted by ${\mathbf{d}}_i$. Accordingly, we denote the set of BS locations as $\Phi =\{{\mathbf{d}}_i,i\in \mathbb{N}\}$. In practical scenarios, each BS typically serves more than tens of users, as the density of users is much higher than that of BSs. Therefore, we assume that the number of users within the Voronoi region of each BS is much larger than $N_t$. Since our primary focus is on designing a distance metric for quantization, rather than on gaining advantages through specific user selection, we do not consider a particular user selection strategy. As a result, each BS randomly selects K users from its Voronoi region, where it is assumed that $2\le K\le N_t/N_r$. The number of users, K, is restricted to be less than or equal to $N_t/N_r$ because we use BD as a linear precoding scheme. More specifically, BD is a precoding method designed to eliminate multiuser interference by exploiting the null spaces of different channels. However, it has the fundamental limitation that complete elimination is possible only when $K\le N_t/N_r$, even with perfect CSIT.

Our target performance metric is the ergodic rate achieved by an arbitrary user in the network. Due to the symmetry of random environments, ergodic rates of users are equivalent, assuming that the same transmission and reception strategies are employed by BSs and users. Additionally, a homogeneous PPP is a stationary process such that we can assume that the target user is located at the origin without loss of generality. For simplicity, we denote the target user as index 1 and the serving BS of user 1 as $b_1\in \mathbb{N}$. In other words, $b_1$ is the index of the BS that is closest to user 1. Furthermore, the set of K users served by $b_1$ is denoted by $\mathcal{K}=\{1,\dots ,K\}$.

The wireless fading channel between the serving BS and the target user is denoted by the complex matrix $\mathbf{H}\in {\mathbb{C}}^{N_t\times N_r}$, and the channel between BS $i\ne b_1$ and the target user is represented by the complex matrix ${\mathbf{G}}_i\in {\mathbb{C}}^{N_t\times N_r}$. We assume Rayleigh fading, where the entries of $\mathbf{H}$ and ${\mathbf{G}}_i$ follow independent and identically distributed (i.i.d.) circularly symmetric complex Gaussian distributions with variances normalized to 1. The precoding matrices of BS $b_1$ are denoted by ${\mathbf{V}}_1,\dots ,{\mathbf{V}}_K$, and information symbol vectors of BS $b_1$ are denoted by ${\mathbf{s}}_1,\dots ,{\mathbf{s}}_K$; ${\mathbf{V}}_l$ and ${\mathbf{s}}_l$ represent precoding matrix and information vector prepared for user $l\in \mathcal{K}$, respectively. The precoding matrices and information symbol vectors of BS $i\ne b_1$ are denoted by ${\mathbf{F}}_{i,1},\dots ,{\mathbf{F}}_{i,K}$ and ${\mathbf{t}}_{i,1},\dots ,{\mathbf{t}}_{i,K}$, respectively. Consequently, the received signal at user 1 is modeled as follows:

$$\begin{array}{cc}\hfill \mathbf{y}& ={\parallel \mathbf{d}\parallel }^{-{\displaystyle \frac{\alpha }{2}}}{\mathbf{H}}^H{\mathbf{V}}_1{\mathbf{s}}_1+\sum _{l=2}^K{\parallel \mathbf{d}\parallel }^{-{\displaystyle \frac{\alpha }{2}}}{\mathbf{H}}^H{\mathbf{V}}_l{\mathbf{s}}_l+\sum _{i=1,i\ne b_1}^{\infty }\sum _{j=1}^K{\parallel {\mathbf{d}}_i\parallel }^{-{\displaystyle \frac{\alpha }{2}}}{\mathbf{G}}_i^H{\mathbf{F}}_{i,j}{\mathbf{t}}_{i,j}+\mathbf{z},\hfill \end{array}$$

(1)

where the distance between user 1 and BS $b_1$ is denoted by $\mathbf{d}$ for simplicity, i.e., $\mathbf{d}={\mathbf{d}}_{b_1}$. The vector $\mathbf{z}$ represents the additive white Gaussian noise (AWGN) at user 1, with entries that are i.i.d. circularly symmetric complex Gaussian random variables, each with variance normalized to 1. The path-loss exponent $\alpha$ is assumed to be greater than two. To primarily concentrate on the performance achieved through precoding based on quantized CDI, we assume equal power allocation, consistent with previous studies in the literature [7,8]; i.e., $\mathbb{E}\left[{\mathbf{s}}_1{\mathbf{s}}_1^H\right]={\displaystyle \frac{P}{K{N}_r}}{\mathbf{I}}_{N_r}$, where P denotes the total transmit power.

2.2. Precoding Matrix Construction Based on BD

In MU-MIMO downlink systems, it is challenging for users to perform effective signal processing to eliminate interference from signals intended for other users. This difficulty arises primarily because joint signal processing among different users is practically infeasible. Consequently, essential signal processing for interference cancellation is typically performed at the transmitter side, and the linear precoding is a typical example that accomplishes this purpose. Among various linear precoding techniques, we employ BD, which is specifically designed to eliminate multiuser interference. If the transmitter has perfect CSI, BD can completely eliminate multiuser interference, resulting in near-optimal performance in the high SNR regime. In this subsection, we briefly summarize the process of deriving BD precoding matrix for user 1 (i.e., the target user), and the precoding matrices for the other users are obtained using the same procedure.

Since the purpose of BD is to completely eliminate multiuser interference, the precoding matrix of user 1 must be chosen to satisfy that the signal term of user 1 will be eliminated at the other users received signal. For ease of description, we may consider the following auxiliary matrix:

$$\begin{array}{c}\hfill \mathbf{U}={[{\mathbf{H}}_2,\dots ,{\mathbf{H}}_K]}^H\in {\mathbb{C}}^{N_r(K-1)\times N_t},\end{array}$$

(2)

where ${\mathbf{H}}_k$ denotes the channel matrices between user k and BS $b_1$, for $k=2,\dots ,K$. Then, by choosing ${\mathbf{V}}_1$ (which is the precoding matrix of user 1) as a matrix that lies within the null space of $\mathbf{U}$, the signal intended for user 1 will be completely eliminated from the received signals of other users. Given the freedom to select any matrix within the null space of $\mathbf{U}$, it is preferable to choose a matrix that maximizes the capacity of each user’s downlink rate, provided that the precoding matrix belongs to the null space of $\mathbf{U}$. To achieve this, we obtain the matrix ${\mathbf{A}}_1\in {\mathbb{C}}^{N_t\times (N_t-N_r(K-1))}$, whose columns consist of the orthonormal basis of the null space of $\mathbf{U}$ (this can be easily obtained using singular value decomposition (SVD) or QR decomposition). Subsequently, the BD matrix ${\mathbf{V}}_1\in {\mathbb{C}}^{N_t\times N_r}$ for user 1 is obtained by using the orthonormal basis of ${\mathbf{H}}^H{\mathbf{A}}_1$ as the columns of ${\mathbf{V}}_1$.

The BD process described thus far is feasible only when the transmitter has perfect CSIT. However, since this is difficult to achieve in practical systems, the transmitter typically uses estimates of the channel matrices to construct BD matrices. Let $\widehat{\mathbf{H}}$ denote the estimate of $\mathbf{H}$, and ${\widehat{\mathbf{H}}}_k$ denote the estimate of ${\mathbf{H}}_k$, for $k=2,\dots ,K$, used for BD matrices construction. Then, ${\mathbf{V}}_1$ is obtained by following the process in this subsection, with $\mathbf{H}$, ${\mathbf{H}}_2,\dots ,{\mathbf{H}}_K$ replaced by their respective estimates $\widehat{\mathbf{H}}$, ${\widehat{\mathbf{H}}}_2,\dots ,{\widehat{\mathbf{H}}}_K$. These estimates are derived through finite-rate quantization by the users with the corresponding channels, and each user feeds back the corresponding information to the transmitter. The following subsection details this process.

2.3. Finite-Rate Feedback Model

We describe how user 1 (i.e., the target user) quantizes and feeds back its channel, with the other users follow the same process. A codebook-based quantization method is used to quantize the channel matrix $\mathbf{H}$. Specifically, both BS $b_1$ and user 1 share a pre-engaged codebook, which in this study is defined as $\mathcal{C}=\{{\mathbf{W}}_1,\dots ,{\mathbf{W}}_{2^B}\}$, where B represents the number of feedback bits per user. The quantization index is then fed back to the transmitter via the uplink channel. As BD eliminates multiuser interference by utilizing the intersection of the nullspaces of the other channels, each codeword (i.e., the element of the codebook) is sufficient to be a semi-unitary matrices in ${\mathbb{C}}^{N_t\times N_r}$. That is, we assume that ${\mathbf{W}}_j^H{\mathbf{W}}_j={\mathbf{I}}_{N_r}$. This assumption is consistent with the approaches in the relevant literature [8,22].

The essential concept of codebook-based quantization is to find the codeword in $\mathcal{C}$ that is closest to the $\mathbf{H}$. Since the distance between two matrices can be measured in various ways, determining an appropriate distance measure is important. When $N_r=1$, chordal distance is commonly used [7] as the distance measure because a decrease in chordal distance implies an increase in the achievable rate. Although the chordal distance is still widely used when $N_r>1$ due to its intuitive performance [8], it does not guarantee optimal performance. Surely, the optimal distance measure depends on the specific performance metric being considered. In this study, where the main performance metric is the achievable rate of user 1, we consider an estimate of this rate and use the minus value of this estimate as the distance measure. This approach ensures that the quantization process can maximizes the approximated rate of each user. In this context, the quantization process can be summarized as follows:

$$\begin{array}{c}\hfill \widehat{n}=\underset{j\in \mathcal{J}}{argmin}d\left({\mathbf{W}}_j,\mathbf{H}\right),\end{array}$$

(3)

where $\mathcal{J}=\{1,\dots ,2^B\}$ represents the index set of the codebook, and $d(·,·)$ denotes the distance measure. Accordingly, we use the resulting quantized channel as the estimate of $\mathbf{H}$, which is then used to construct the BD matrices as described in the previous subsection. Thus, we assume that

$$\begin{array}{c}\hfill \widehat{\mathbf{H}}={\mathbf{W}}_{\widehat{n}}.\end{array}$$

(4)

The index $\widehat{n}\in \mathcal{J}$ is fed back to BS $b_1$ to facilitate the construction of the BD precoding matrices.

2.4. Performance Measure

For simplicity, we adopt the following notations throughout the paper:

$$\begin{array}{c}{\Psi }_U\triangleq \sum _{l=2}^K{\mathbf{H}}^H{\mathbf{V}}_l{\mathbf{V}}_l^H\mathbf{H},{\Psi }_{C,N}\triangleq {\parallel \mathbf{d}\parallel }^{\alpha }\sum _{i=1,i\ne b_1}^{\infty }\parallel {\mathbf{d}}_i{\parallel }^{-\alpha }\sum _{j=1}^K{\mathbf{G}}_i^H{\mathbf{F}}_{i,j}{\mathbf{F}}_{i,j}^H{\mathbf{G}}_i+{\parallel \mathbf{d}\parallel }^{\alpha }{\displaystyle \frac{K{N}_r}{P}}.\end{array}$$

(5)

The instantaneous rate in limited-feedback-based downlink spatial division multiple access is bounded by the following value for given channel realizations [7,8,20]:

$$\begin{array}{c}{R}_I\triangleq {log}_2{\displaystyle \frac{\det \left({\mathbf{H}}^H{\mathbf{V}}_1{\mathbf{V}}_1^H\mathbf{H}+{\Psi }_U+{\Psi }_{C,N}\right)}{\det \left({\Psi }_U+{\Psi }_{C,N}\right)}}\end{array}$$

(6)

$$\begin{array}{cc}\hfill & ={log}_2\det ({\mathbf{H}}^H{\mathbf{V}}_1{{\mathbf{V}}_1}^H\mathbf{H}+{\Psi }_U+{\Psi }_{C,N})-{log}_2\det ({\Psi }_U+{\Psi }_{C,N}).\hfill \end{array}$$

(7)

The primary performance metric of this study is the ergodic achievable rate of user 1, which is defined as follows:

$$\begin{array}{c}\hfill R_E=\mathbb{E}\left[R_I\right].\end{array}$$

(8)

2.5. Preliminaries

In this study, we denote the compact SVD of $\mathbf{H}$ as

$$\begin{array}{c}\hfill \mathbf{H}=\tilde{\mathbf{H}}{\mathbf{\Sigma }}^{{\displaystyle \frac{1}{2}}}{\mathbf{U}}^H,\end{array}$$

(9)

where $\tilde{\mathbf{H}}\in {\mathbb{C}}^{N_t\times N_r}$, $\mathbf{U}\in {\mathbb{C}}^{N_r\times N_r}$, and $\mathbf{\Sigma }\in {\mathbb{C}}^{N_r\times N_r}$. We define ${\mathcal{CW}}_m(a,\Omega )$ as the complex Wishart distribution and ${\mathcal{CB}}_m(a,b)$ as the complex matrix variate beta distribution. From [8,24,25,26,27], we know that

$$\begin{array}{cc}\hfill & {\mathbf{H}}^H\mathbf{H}\stackrel{d}{=}{\mathcal{CW}}_{N_r}(N_t,{\mathbf{I}}_{N_r}),\hfill \end{array}$$

(10)

$$\begin{array}{cc}\hfill & {\mathbf{H}}^H{\mathbf{W}}_j{\mathbf{W}}_j^H\mathbf{H}\stackrel{d}{=}{\mathcal{CW}}_{N_r}(N_r,{\mathbf{I}}_{N_r}),\hfill \end{array}$$

(11)

$$\begin{array}{cc}\hfill & {\mathbf{H}}^H({\mathbf{I}}_{N_t}-{\mathbf{W}}_j{\mathbf{W}}_j^H)\mathbf{H}\stackrel{d}{=}{\mathcal{CW}}_{N_r}(N_t-N_r,{\mathbf{I}}_{N_r}),\hfill \end{array}$$

(12)

where $\stackrel{d}{=}$ denotes the equality in distribution. From Equation (5.1.7) of [25], it follows that

$$\begin{array}{c}\hfill {\tilde{\mathbf{H}}}^H({\mathbf{I}}_{N_t}-{\mathbf{W}}_j{\mathbf{W}}_j^H)\tilde{\mathbf{H}}\stackrel{d}{=}{\mathcal{CB}}_{N_r}(N_t-N_r,N_r),\end{array}$$

(13)

for each $j\in \mathcal{J}$.

3. Proposed Distance Metric

The primary objective of this study is to design an appropriate distance measure for quantization that is suitable for limited-feedback-based BD in dense cellular networks. As discussed in Section 2.3, simply using well-known measures like the chordal distance may not be optimal depending on the cellular environments. We demonstrate that a proper approximation of the instantaneous rate $R_I$ can achieve significantly higher rates compared to typical distance measures between two matrices. Similar studies on limited-feedback-based BD, such as those in [10], have been proposed; however, these schemes generally did not account for the effects of densely designed cellular networks.

Proposed Distance Measure

Ideally, the optimal distance measure would correspond to assuming $d(\mathbf{H},{\mathbf{W}}_j)=-R_I$ in (3), as we would directly maximize the user rate based on this assumption. However, $R_I$ consists of many information not available to each user at the time of channel quantization. This unknown information includes all precoding matrices and the channel matrices of all other users, which are necessary for calculating $R_I$. Therefore, we need to approximate these unknown values, and we want to replace them with their expected values, as they are represented by random variables.

Using each ${\mathbf{W}}_j\in \mathcal{C}$, the left unitary matrix of user 1’s channel can be decomposed as [8,22]

$$\begin{array}{cc}\hfill \tilde{\mathbf{H}}& ={\mathbf{W}}_j{\mathbf{W}}_j^H\tilde{\mathbf{H}}+\left({\mathbf{I}}_{N_t}-{\mathbf{W}}_j{\mathbf{W}}_j^H\right)\tilde{\mathbf{H}}.\hfill \end{array}$$

(14)

In (14), the first term at the right-hand side corresponds to the component of $\tilde{\mathbf{H}}$ that belongs to the column space of the codeword, while the second term corresponds to the component that lies in the left-null space. Let the compact SVD of the second term be denoted as

$$\begin{array}{cc}\hfill & \left({\mathbf{I}}_{N_t}-{\mathbf{W}}_j{\mathbf{W}}_j^H\right)\tilde{\mathbf{H}}={\mathbf{S}}_j{\mathbf{\Lambda }}_j^{{\displaystyle \frac{1}{2}}}{\mathbf{E}}_j^H.\hfill \end{array}$$

(15)

The semi-unitary matrix ${\mathbf{S}}_j\in {\mathbb{C}}^{N_t\times N_r}$ is isotropically distributed within the left null space of ${\mathbf{W}}_j$, and ${\mathbf{\Lambda }}_j^{{\displaystyle \frac{1}{2}}}\in {\mathbb{C}}^{N_r\times N_r}$ is a diagonal matrix whose entries are nonnegative singular values arranged in descending order. If all elements of ${\mathbf{\Lambda }}_j$ are zero, then the codeword ${\mathbf{W}}_j$ is equal to $\tilde{\mathbf{H}}$. Therefore, a well-designed distance measure must efficiently suppress the magnitude of the elements in ${\mathbf{\Lambda }}_j$. A well-known example of such a measure is the chordal distance, defined as the trace of ${\mathbf{\Lambda }}_j$ [8]. However, distance measures that consider only the elements of ${\mathbf{\Lambda }}_j$, such as the chordal distance, disregard channel gain information, which can significantly affect the achievable rate. This is the reason why we attempt to obtain an approximation to the instantaneous rate $R_I$.

To this end, we individually approximate the terms ${\Psi }_U$ and ${\Psi }_{C,N}$ in (5). By combining (5), (9), and (15), ${\Psi }_U$ can be expressed as follows for each codeword ${\mathbf{W}}_j$:

$$\begin{array}{cc}\hfill {\Psi }_U& =\sum _{l=2}^K\mathbf{U}{\mathbf{\Sigma }}^{{\displaystyle \frac{1}{2}}}{\mathbf{E}}_j{\mathbf{\Lambda }}_j^{{\displaystyle \frac{1}{2}}}{\mathbf{S}}_j^H{\mathbf{V}}_l{\mathbf{V}}_l^H{\mathbf{S}}_j{\mathbf{\Lambda }}_j^{{\displaystyle \frac{1}{2}}}{\mathbf{E}}_j^H{\mathbf{\Sigma }}^{{\displaystyle \frac{1}{2}}}{\mathbf{U}}^H.\hfill \end{array}$$

(16)

Since ${\mathbf{V}}_l$ for $l=2,\dots ,K$ cannot be known by user 1 (the target user), we approximate the projection matrix ${\mathbf{S}}_j^H{\mathbf{V}}_l{\mathbf{V}}_l^H{\mathbf{S}}_j$ using its expected value. Given that it follows a complex matrix-variate beta distribution with parameters $N_r$ and $N_t-2N_r$ [8], we have

$$\begin{array}{c}\hfill \mathbb{E}\left[{\mathbf{S}}_j^H{\mathbf{V}}_l{\mathbf{V}}_l^H{\mathbf{S}}_j\right]={\displaystyle \frac{N_r}{N_t-N_r}}{\mathbf{I}}_{N_r}.\end{array}$$

(17)

Replacing ${\mathbf{S}}_j^H{\mathbf{V}}_l{\mathbf{V}}_l^H{\mathbf{S}}_j$ in (16) with this expectation yields the following approximation:

$$\begin{array}{c}{\Psi }_U\approx {\displaystyle \frac{N_r}{N_t-N_r}}\sum _{l=2}^K\mathbf{U}{\mathbf{\Sigma }}^{{\displaystyle \frac{1}{2}}}{\mathbf{E}}_j{\mathbf{\Lambda }}_j{\mathbf{E}}_j^H{\mathbf{\Sigma }}^{{\displaystyle \frac{1}{2}}}{\mathbf{U}}^H\triangleq {\widehat{\Psi }}_U.\end{array}$$

(18)

This approximated value can be fully calculated by each user when quantizing its channel. Next, we approximate ${\Psi }_{C,N}$, which is defined in (5) as follows:

$$\begin{array}{c}\hfill {\Psi }_{C,N}={\parallel \mathbf{d}\parallel }^{\alpha }\sum _{i=1,i\ne b_1}^{\infty }\parallel {\mathbf{d}}_i{\parallel }^{-\alpha }\sum _{j=1}^K{\mathbf{G}}_i^H{\mathbf{F}}_{i,j}{\mathbf{F}}_{i,j}^H{\mathbf{G}}_i+{\parallel \mathbf{d}\parallel }^{\alpha }{\displaystyle \frac{K{N}_r}{P}}{\mathbf{I}}_{N_r}.\end{array}$$

(19)

In this formula, user 1 lacks information about the matrix ${\mathbf{G}}_i^H{\mathbf{F}}_{i,j}{\mathbf{F}}_{i,j}^H{\mathbf{G}}_i$, so it will be replaced with its expected value (although it is technically possible for user 1 to estimate the channel ${\mathbf{G}}_i$ from BS $i\ne b_1$, it is impractical to estimate channels from unassociated BSs). As ${\mathbf{F}}_{i,j}$ is the BD matrix of BS $i\ne b_1$, it is represented by a semi-unitary matrix, as discussed in Section 2.2. Consequently, ${\mathbf{F}}_{i,j}{\mathbf{F}}_{i,j}^H$ is hermitian and idempotent, so according to Lemma 1 in [24], we have

$$\begin{array}{c}\hfill {\mathbf{G}}_i^H{\mathbf{F}}_{i,j}{\mathbf{F}}_{i,j}^H{\mathbf{G}}_i\stackrel{d}{=}{\mathcal{CW}}_{N_r}(N_r,{\mathbf{I}}_{N_r}),\end{array}$$

(20)

and thus

$$\begin{array}{c}\hfill \mathbb{E}\left[{\mathbf{G}}_i^H{\mathbf{F}}_{i,j}{\mathbf{F}}_{i,j}^H{\mathbf{G}}_i\right]=N_r{\mathbf{I}}_{N_r}.\end{array}$$

(21)

Moreover, the multicell interference term in (19) is a summation of infinitely many signals, which user 1 cannot evaluate at the quantization stage. Fortunately, the impact of interference rapidly decreases with increasing distance from the target user. Therefore, when approximating ${\Psi }_{C,N}$, we consider only a finite number of BSs, where such BSs are selected in ascending order of the distance from user 1. Let $b_i$ denote the index of i-th nearest BS to user 1. Then, using (20) and (21), we approximate ${\Psi }_{C,N}$ as

$$\begin{array}{c}\hfill {\Psi }_{C,N}\approx \left(\underset{I_1}{\underbrace{{\parallel \mathbf{d}\parallel }^{\alpha }\sum _{i=2}^M{\parallel {\mathbf{d}}_{b_i}\parallel }^{-\alpha }K{N}_r}}+\underset{I_2}{\underbrace{{\parallel \mathbf{d}\parallel }^{\alpha }\sum _{i=M+1}^{\infty }{\parallel {\mathbf{d}}_{b_i}\parallel }^{-\alpha }K{N}_r}}+{\parallel \mathbf{d}\parallel }^{\alpha }{\displaystyle \frac{K{N}_r}{P}}\right){\mathbf{I}}_{N_r},\end{array}$$

(22)

where the distance terms are rearranged in ascending order. We assume that $I_1$, consisting of interference from the $M-1$ nearest BSs, is the dominant interfering term, while the remaining cellular interference in $I_2$ is less important. We propose two types of approximations for $R\left(B\right)$ based on the treatment of $I_2$. The first approximation completely ignores $I_2$, and the second replaces it with its expected value, which is obtained by using

$$\begin{array}{cc}\hfill \mathbb{E}[\sum _{i=M+1}^{\infty }\parallel {\mathbf{d}}_{b_i}{\parallel }^{-\alpha }|{\mathbf{d}}_{b_M}]& \stackrel{\left(a\right)}{=}{\int }_{\sqrt{x^2+y^2}>{\mathbf{d}}_{b_M}}{\left(\sqrt{x^2+y^2}\right)}^{-\alpha }\lambda dxdy\hfill \\ \hfill & \stackrel{\left(b\right)}{=}{\int }_0^{2\pi }{\int }_{{\mathbf{d}}_{b_M}}^{\infty }{r}^{-\alpha }\lambda rdrd\theta \hfill \\ \hfill & ={\displaystyle \frac{2\pi \lambda }{\alpha -2}}{\parallel {\mathbf{d}}_{b_M}\parallel }^{2-\alpha },\hfill \end{array}$$

(23)

where $\left(a\right)$ follows Campbell’s theorem and $\left(b\right)$ follows the conversion from Cartesian coordinate to Polar coordinate. In this context, we propose the following two approximations for ${\Psi }_{C,N}$:

$$\begin{array}{cc}\hfill {\widehat{\Psi }}_1& \triangleq \left({\parallel \mathbf{d}\parallel }^{\alpha }\sum _{i=2}^M\parallel {\mathbf{d}}_{b_i}{\parallel }^{-\alpha }K{N}_r+{\parallel \mathbf{d}\parallel }^{\alpha }{\displaystyle \frac{K{N}_r}{P}}\right){\mathbf{I}}_{N_r},\hfill \end{array}$$

(24)

and

$$\begin{array}{cc}\hfill {\widehat{\Psi }}_2& \triangleq \left({\parallel \mathbf{d}\parallel }^{\alpha }\sum _{i=2}^M\parallel {\mathbf{d}}_{b_i}{\parallel }^{-\alpha }K{N}_r+\mathbb{E}\left[I_2\right]+{\parallel \mathbf{d}\parallel }^{\alpha }{\displaystyle \frac{K{N}_r}{P}}\right){\mathbf{I}}_{N_r}\hfill \\ \hfill & =K{N}_r{\parallel \mathbf{d}\parallel }^{\alpha }\left(\sum _{i=2}^M\parallel {\mathbf{d}}_{b_i}{\parallel }^{-\alpha }+{\displaystyle \frac{2\pi \lambda }{\alpha -2}}{\parallel {\mathbf{d}}_{b_M}\parallel }^{2-\alpha }+{\displaystyle \frac{1}{P}}\right){\mathbf{I}}_{N_r}.\hfill \end{array}$$

(25)

Using ${\widehat{\Psi }}_U$ from (18) to approximate ${\Psi }_U$, and using either of ${\widehat{\Psi }}_1$ or ${\widehat{\Psi }}_2$ to approximate ${\Psi }_{C,N}$, we propose the following two approximations for $R_I$:

$$\begin{array}{c}{\widehat{R}}_1\triangleq {log}_2{\displaystyle \frac{\det \left({\mathbf{H}}^H{\mathbf{V}}_1{\mathbf{V}}_1^H\mathbf{H}+{\widehat{\Psi }}_U+{\widehat{\Psi }}_1\right)}{\det \left({\widehat{\Psi }}_U+{\widehat{\Psi }}_1\right)}},\end{array}$$

(26)

and

$$\begin{array}{c}{\widehat{R}}_2\triangleq {log}_2{\displaystyle \frac{\det \left({\mathbf{H}}^H{\mathbf{V}}_1{\mathbf{V}}_1^H\mathbf{H}+{\widehat{\Psi }}_U+{\widehat{\Psi }}_2\right)}{\det \left({\widehat{\Psi }}_U+{\widehat{\Psi }}_2\right)}}.\end{array}$$

(27)

Our approximations in (26) and (27) involve replacing certain matrix components that contribute to the achievable rate with their expected values. While using the exact values of these matrices would yield optimal performance, these approximations are necessary because the target user cannot access these values when performing quantization for limited feedback.

It should be noted that, under our assumption, user 1 needs to know the locations of M BSs, including its serving BS $b_1$. Therefore, using excessively large values for M would be impractical. Consequently, the two proposed distance measures are defined as

$$\begin{array}{cc}\hfill & d_1(\mathbf{H},{\mathbf{W}}_j)\triangleq -{\widehat{R}}_1,\hfill \\ \hfill & d_2(\mathbf{H},{\mathbf{W}}_j)\triangleq -{\widehat{R}}_2,\hfill \end{array}$$

(28)

to maximize the approximated rates through quantization process defined in (3), respectively.

4. Simulation Results and Discussion

In this section, we verify the performance of the proposed distance measures using Monte Carlo simulations with random parameters in our network model. The primary performance metric for comparison is the ergodic achievable rate $R_E$, as defined in (8). We compare the proposed distance measures with the following two benchmark distance measures.

4.1. Benchmark Distance Measures

Previous studies have proposed distance measures that aim to efficiently suppress the singular values of quantization error matrix, specifically the diagonal entries of ${\mathbf{\Lambda }}_j^{{\displaystyle \frac{1}{2}}}$, as described in Section 3. We consider two typical examples of such measures as benchmark schemes for performance comparison.

The first example, referred to as benchmark scheme 1 in this section, uses the (1,1)-th element of ${\mathbf{\Lambda }}_j$ as the distance measure. That is, we define

$$\begin{array}{c}\hfill d_{LS}({\mathbf{W}}_j,\mathbf{H})\triangleq {\left[{\mathbf{\Lambda }}_j\right]}_{1,1}.\end{array}$$

(29)

For quantization, $d({\mathbf{W}}_j,\mathbf{H})=d_{LS}({\mathbf{W}}_j,\mathbf{H})$ is used in (3) when applying benchmark scheme 1. Note that ${\left[{\mathbf{\Lambda }}_j\right]}_{1,1}$ represents the square of the largest singular value of the quantization error matrix. Since the singular values are arranged in descending order, suppressing ${\left[{\mathbf{\Lambda }}_j\right]}_{1,1}$ corresponds to suppressing all diagonal elements of ${\mathbf{\Lambda }}_j$. By employing $d_{LS}$ as a distance measure, the quantization error can be approximated using a QUB model [22]. Consequently, this measure is useful for basic performance analysis. However, $d_{LS}$ is not an optimal distance measure because it only indirectly suppress the other components of the quantization error matrix, aside from the $(1,1)$-th element. An alternative is to use a distance measure that uniformly suppresses all diagonal entries of the quantization error matrix. A well-known example of such a measure is the chordal distance, defined as [8]

$$\begin{array}{c}\hfill d_C({\mathbf{W}}_j,\mathbf{H})\triangleq \mathrm{tr}\left({\mathbf{\Lambda }}_j\right)=N_r-\mathrm{tr}\left({\tilde{\mathbf{H}}}^H{\mathbf{W}}_j{\mathbf{W}}_j^H\tilde{\mathbf{H}}\right).\end{array}$$

(30)

The quantization method using chordal distance (i.e., assume $d({\mathbf{W}}_j,\mathbf{H})=d_C({\mathbf{W}}_j,\mathbf{H})$ in (3)) is referred to as benchmark scheme 2 in this section.

4.2. Simulation Results

This subsection presents Monte Carlo simulation results for the ergodic achievable rate $R_E$ to demonstrate the importance of using an appropriate approximation for the signal to interference plus noise ratio (SINR) as a distance measure, particularly when precoding is applied with $N_r>1$. The random components, including BS locations, channel distributions, and precoding designs, are generated based on the definitions described in Section 2. For codebook design, we use a randomly generated codebook, rather than explicitly designing an optimal codebook. This approach is justified because a random codebook can provide performance close to that of an optimally designed codebook, unless the number of quantization bits B is too small [7,8,24]. For details on the random generation of each codeword, readers are referred to [8].

The number of trials must be sufficiently large to obtain a reliable estimate of $R_E$, and we averaged results over at least $\mathrm{10,000}$ trials for each simulation. The positions and number of BSs in the each trial are governed by a Poisson distribution with the associated density. Subsequently, the BSs are uniformly distributed within a two-dimensional circular region, with a radius of 5 km for simplicity. As described in Section 2, the target user can be assumed to be located at the origin without loss of generality. The closest BS to the target user is considered the serving BS and is denoted by the index $b_1$. We assume that $\alpha =4$ for all simulations. In this section, the proposed schemes 1 and 2 correspond to using $d_1$ and $d_2$ as the distance measures in (3), respectively.

In Figure 1, we depict the ergodic rate of user 1 ($R_E$) as a function of the SNR. The rates achieved by the proposed schemes are compared with those of the benchmark schemes, and the proposed methods generally outperform the conventional approaches. The rate converges as the SNR increases due to multiuser and multicell interference, and the advantage of the proposed methods becomes more pronounced as the SNR increases. Both proposed schemes achieve similar performance, as we used a relatively large value of $M=10$.

To further assess the impact of M, which represents the number of BSs used to estimate the approximated SINRs in the proposed schemes, Figure 2 compares the ergodic rate of user 1 with respect to M. Since M is utilized exclusively by the proposed methods, the benchmark schemes exhibit consistent performance and generally show worse performance compared to the proposed schemes. The proposed scheme 1, which disregards interference from BSs beyond $b_M$, is less accurate than the proposed scheme 2 when M is very small. However, as M increases, the performance gap between the two proposed schemes diminishes, indicating that the interference from the M nearest BSs becomes dominant compared to that from more distant BSs when M is sufficiently large.

In Figure 3, the sum rate ($K{R}_E$) of the proposed schemes is compared with that of the benchmark schemes as a function of the number of transmit antennas. Similar to the results presented in Figure 1 and Figure 2, the proposed schemes consistently achieve significantly higher rates than the benchmark schemes. Note that the sum rate decreases as the number of users for BD increases, which is attributed to the increased multiuser interference caused by a larger number of users with a fixed value of B. Nonetheless, each BS can serve more users simultaneously by employing BD within the same time and frequency grid. Additionally, this reduction in sum rate can be mitigated by utilizing a greater number of feedback bits when there is a larger number of users. Figure 4 illustrates the impact of the number of feedback bits on performance. As observed in previous results, the proposed methods consistently outperform the benchmark schemes. Moreover, the performance gap widens as the number of feedback bits increases, as the use of a more accurate distance measure allows for greater benefits from the quantization process.

5. Conclusions

This study demonstrates that the proposed distance metrics for limited-feedback-based precoding significantly enhance spectral efficiency in MU-MIMO systems, particularly in dense cellular networks. Monte Carlo simulations confirm that these metrics consistently outperform conventional methods in terms of ergodic rate, especially at higher SNR levels. The adaptability of the proposed methods is evident as their performance remains robust across varying numbers of BS distances considered in the SINR approximation. Additionally, the proposed schemes generally exhibit superior sum rate performance compared to benchmark schemes, regardless of system parameters such as the number of scheduled users, the number of transmit and receive antennas, etc. This suggests that our approach effectively mitigates both multiuser and multicell interferences while supporting multiple users simultaneously. The results highlight the potential of these distance metrics to improve feedback and precoding strategies in next-generation wireless systems.

Note that our analytical results are valid only when block diagonalization (BD) is specifically used in our system model, which assumes uncorrelated transmit antennas in a dense cellular network modeled by a homogeneous PPP. However, the concept of using the expected SINR as a distance metric can be extended to other precoding schemes if an appropriate SINR estimate can be computed at the user. Thus, the idea could be generalized to scenarios involving different precoders, such as regularized BD, or alternative network models, such as cell-free MIMO systems [28].