Spatio-Radio Resource Management and Hybrid Beamforming for Limited Feedback Massive MIMO Systems

Abstract

In this paper, a joint spatio–radio frequency resource allocation and hybrid beamforming scheme for the massive multiple-input multiple-output (MIMO) systems is proposed. We consider limited feedback two-stage hybrid beamformimg for decomposing the precoding matrix at the base-station. To reduce the channel state information (CSI) feedback of massive MIMO, we utilize the channel covariance-based RF precoding and beam selection. This beam selection process minimizes the inter-group interference. The regularized block diagonalization can mitigate the inter-group interference, but requires substantial overhead feedback. We use channel covariance-based eigenmodes and discrete Fourier transforms (DFT) to reduce the feedback overhead and design a simplified analog precoder. The columns of the analog beamforming matrix are selected based on the users’ grouping performed by the K-mean unsupervised machine learning algorithm. The digital precoder is designed with joint optimization of intra-group user utility function. It has been shown that more than $50\%$ feedback overhead is reduced by the eigenmodes-based analog precoder design. The joint beams, users scheduling and limited feedbacK-based hybrid precoding increases the sum-rate by $27.6\%$ compared to the sum-rate of one-group case, and reduce the feedback overhead by $62.5\%$ compared to the full CSI feedback.

1. Introduction

The scarcity of available frequency band for wireless communications has led to the inclusion of millimeter Wave (mmWave) frequencies in cellular communications. This has opened the doors for massive multiple-input multiple-output (MIMO) systems. Due to high transmission frequencies, fabrication of large number of antennas with small form factor has become possible. MmWave band has inherent hindrances, like, high path-loss and absorption-loss. It has been known that MIMO systems advantages (spatial multiplexing or diversity gain) are scaled-up with the number of antennas. In summary, one can enjoy the benefits of the large bandwidth available at mmWave frequencies by combating high path and absorption losses with massive MIMO directional beamforming. Future mmWave massive MIMO-based cellular networks will be as shown in Figure 1. Due to the high pathloss on one hand and high directional gain on the other hand, the inter-cell interference and cell boundaries will become meaningless. The fixed area size cell boundaries of traditional cellular will probably no longer exist in the future mmWave massive MIMO systems. Narrow beams can serve distant user equipment (UE) without interfering other UEs provided that there is no obstacle between BS and intended UE, whereas a closely located UE may deprive of connection due to the obstacles.

The cost of massive MIMO is in terms of excessive feedback overhead for channel estimation along with the hardware complexity of RF chains (increased number of radio-frequency (RF) chains). The feedback overhead has been tackled separately for frequency division duplex (FDD) and time division duplex (TDD) systems. In FDD systems, the uplink channel estimation consists of fewer overheads compared to the downlink channel estimation, because generally, the number of transmit antennas $N_t$ is larger than the number of users K, and the number of receive antennas per user $n_k$ ($N_t\gg K$ and $N_t\gg n_k$). The most common technique to reduce the downlink channel estimation overhead is joint spatial division multiplexing (JSDM) [1]. The JSDM uses two-stage precoding: second order channel statistics (covariance)-based user grouping and the traditional MU-MIMO linear precoding (zero-forcing) for the inter-user interference mitigation based on the low-dimensional effective channel. In TDD, only uplink channel estimation is done and the downlink channel estimates are obtained by the transpose of the uplink channel using the channel reciprocity principle. The TDD massive MIMO systems suffer from pilot contamination when the BS receives non-orthogonal pilot signals from the neighboring cells. This pilot contamination degrades the channel estimation and hence, affects both uplink combining and downlink precoding.

In traditional MIMO systems, a separate RF chain (analog-to-digital converter/digital-to-analog converter, serial-to-parallel/parralel-to-serial converter, up/down converter etc) is required for each antenna, but the high power consumption makes it infeasible for the case of massive MIMO systems. Hybrid beamforming technique resolves this problem by dividing the precoding/combining into baseband digital processing and RF analog processing. The  hybrid precoding and combining offer extra degrees of freedom in space domain with a large number of antennas and analog beamforming  [2]. The hybrid beamforming can be realized by using MU-MIMO precoding as baseband digital precoding and the statistical channel state information-based pre-beamforming as RF analog precoding. This limited feedback (due to average CSI) configuration is particularly suited for massive MIMO mmWave systems with a large number of antennas but relatively small number of RF chains [3]. It has been shown [4], that the covariance-based limited feedback works well for mmWave massive MIMO systems, where the number of users is small with respect to the number of BS antennas and the channels are formed by a few multi path components (MPCs) with small angular spread.

Limited work has been done on the joint multiuser massive MIMO resource allocation and hybrid beamforming design. Although mmWave massive MIMO system has a potential of tremendous increase of spectral efficiency. However, the cost and power consumption of power-hungry radio frequency chains (analog-to-digital converter (ADC)/digital-to-analog converter (DAC), parallel to serial converter, serial to parallel converter, up converter/down converter) make it impractical to build a complete RF chain for each antenna. A promising solution to this problem is hybrid beamforming, where the precoder at the transmitter is divided into two parts: analog precoder and digital precoder. The analog precoder (usually a network of phase shifters) at the RF stage reduces the number of RF chains required for the digital precoder. In order to configure these precoders, the transmitter requires channel state information in the form of uplink feedback from users, but in the presence of massive antennas, this feedback becomes a huge load on the wireless uplink, especially in FDD mode. JSDM [4] is a technique used to reduce the feedback overhead. It uses slowly varying average channel statistics to implement the analog precoder; then, the digital precoder is realized by using a low-dimensional effective channel. Till now, different variants of the JSDM have been proposed. Li et al. [5] generalize the JSDM scheme to support non-orthogonal virtual sectorization and with multiple RF chains at both link ends. It uses the Kronecker channel model to decouple the transmit and receive beamforming. Under this channel, the analog beamformer is obtained by stacking strongest eigenbeams of the channel covariance matrix and then the digital beamformer is based on a weighted minimum mean squared error (MMSE) with effective channel. However, the Kronecker model does not characterize the mmWave channel where transmitter and receiver have coupling effects due to highly directional transmission. In [6], the authors apply JSDM using a geometrical channel model and find hybrid precoder and combiner at transmitter and receivers, respectively. Hybrid beamforming with switches (HBwS) has been introduced in [7], where, $L\times N_t$ analog beamformer is controlled by $N_{RF}\times L$ instantaneous CSI based switches. $N_t$ is the number of transmit antennas, $N_{RF}$ is the number of RF chains, and $N_{RF}<L<N_t$. Another switch-based analog beamforming is proposed in [8] but it requires instantaneous CSI for both switching network and the phase shifter network. Also it contains $L=N_t$. The JSDM implementation also requires the training in the downlink to estimate the channel covariance matrix. Most of the work assumes that the CSI is known at both ends. In [9], authors consider the joint optimization of the training resource allocation and channel-statistics-based analog beamformer design by using user centric virtual sectorization. There are different structures for the phase shifter-based analog beamformer, namely, fully connected, sub-connected, and dynamically connected [10]. Park [11] investigate JSDM with these analog beamformer architectures. The dynamic architecture gives better result at the cost of added complexity. In [12], authors propose a hybrid beamforming method with unified analog beamformer by Subspace Construction (SC) based on partial CSI in massive MIMO OFDM system. In [13], statistical CSI based analog beamformer uses regularized block diagonalization to mitigate the inter-group interference and instantaneous CSI based digital beamformer utilizes the weighted MMSE to suppress intra-group interference. Jiang et al. [14] jointly optimize the user selection and beam selection during analog beamforming design. They use Lyapunov-drift optimization framework to obtain the optimal solution. Their work only focuses on the design of statistical CSI based analog precoder and user/beam selection. Our previous work [15] on resource allocation for transmit beamforming develops digital and analog precoders which maximize the sum rate with total power and desired number of RF chains constraints. The provided solutions require full instantaneous CSI at the transmitter and receiver, which, in case of the massive MIMO, consists of large number of pilot transmission in downlink and channel information feedback in the uplink. In this work we exploit the channel similarities by grouping (K-Mean machine learning) the users based on the location information. Low complexity DFT matrix based analog precoder is derived using statistical CSI. This greatly reduces the feedback overhead for the design of zero-forcing digital precoder.

Machine learning (ML) applications for the physical layer of wireless communication systems have been widely reported in [16]. Most of the conventional transmitter and receiver blocks can be replaced by an ML-based auto encoder as suggested by the authors. The large number of antennas in massive MIMO leads to the challenging issue of channel estimation in mmWave communications. A common practice in TDD massive MIMO systems is to utilize the channel reciprocity to get the downlink CSI from uplink channel information estimates. However, in FDD, the channel reciprocity is not applicable and the downlink CSI estimation is very difficult. The downlink channel estimation is known to be hampered by the pilot contamination effect (user to base-station). The quality of channel estimates is deteriorated by the mutual interference caused by the non-orthogonal pilots in a cell. In [17], a supervised learning-based pilot decontamination scheme for massive MIMO uplink is reported. In the proposed ML-based solution, the users’ locations in all cells and the pilot assignments stand for the input features and output labels, respectively. In [18], a deep learning network CsiNet is used to learn the CSI-to-codeword transformation (codebook approach is usually adopted to reduce the feedback overhead) at users’ terminals and inverse CsiNet at base-station. The authors of [19], suggest a learning-based antenna selection for massive MIMO systems. It uses a multiclass K-NN and support vector machine (SVM) for data-driven optimal antenna selection. Wang et al. [20] employs K-nearest neighbor $(K-NN)$ supervised learning for the N beams allocation among K users. In [21], a reinforcement learning based framework for radio resource management in radio access networks has been proposed. In our previous work [22], we used neural networks to reduce the execution time of the computationally intensive resource allocation part of the joint resource allocation and hybrid beamforming design in [15]. However, in this work, we use K-mean based unsupervised machine learning scheme to group the users based on their spatial locations. To the best of our knowledge, there is no research work that jointly consider the spatio–radio resource management and the hybrid beamforming in massive MIMO systems.

In this work, we use spatial channel covariance matrices for the analog beamforming design. We also consider the users to RF beam mapping. This mapping requires channel state information and a search over all possible beam combinations at the base-station. This search is exponential in the number of users [23]. Due to this exponential increase in complexity, we use DFT-based eigenmode beams with RF switches.

Contribution: In this paper, we develop joint spatio–radio resource and hybrid precoding algorithms for limited feedback wideband massive MIMO systems. The contributions of this paper are summarized as follow.

• First, we consider the problem of joint hybrid precoder design with limited feedback and user-beam selection to maximize the sum proportional rate under the total power constraint. The formulated mixed integer programming problem is then transformed to the relaxed-convex optimization problem.
• Second, a low complexity suboptimal solution is provided for the optimization problem. The algorithm generates the analog beamforming matrix, digital beamforming matrix, and the set of users in each group. The DFT/eigenmodes-based analog beamforming is formed using limited statistical CSI feedback from the users. Then, the digital precoder design with users selection is done iteratively.
• Finally, we develop a K-Mean algorithm based unsupervised machine learning scheme for users grouping. These users groups are used to form the limited feedback (statistical channel state information) based analog beamforming matrix. The proposed machine learning based analog beamforming along with the zero-forcing digital precoding and user scheduling gives better performance than the DFT/eigenmodes-based solution.

The rest of the paper is organized as follows. System, signal, and channel model along with the problem formulation are described in Section 2. Section 3 introduce the relaxed-convex transformation of the formulated mixed integer optimization problem. Suboptimal solution to the joint resource allocation and hybrid beamforming based on eigenmodes and discrete Fourier transform is given in Section 4. Section 5 proposes machine learning based users grouping and beam selection for joint optimization problem. Simulation results are given in Section 6, followed by the conclusions in Section 7.

Notations: Bold upper and lower case letters denote vectors and matrices, respectively. The notations ${\mathbf{X}}^{-1}$, ${\mathbf{X}}^{†}$, ${\mathbf{X}}^T$, ${\mathbf{X}}^H$, and $tr\left(\mathbf{X}\right)$ denote the inverse, pseudo-inverse, transpose, Hermitian transpose, and trace of a matrix $\mathbf{X}$. $vec\{·\}$ is a vector operator, $diag\{x_1,\dots ,x_n\}$ is diagonal matrix, and ⊗ is the Kronecker product. ${\parallel ·\parallel }_F$ denotes the Frobenius norm. The $n\times n$ identity matrix is denoted by ${\mathbf{I}}_n$. $\mathbb{E}\{·\}$ represents the expectation with respect to the random variable within the brackets.

2. System Model

Consider a FDD MU-MIMO downlink system where a base station (BS) with $N_t$ antennas is located at the cell center and transmits to K single antenna users as shown in Figure 2. There are G groups of users such that the group $g\in \mathcal{G}=\{1,\dots ,G\}$. Each group contains $K_g$ users.

Assume that the BS and users have the knowledge of the channel. We consider multi-carrier OFDM transmission with narrow-band blocK-fading channel. The BS is equipped with $N_t$ antennas in linear antenna array (ULA) configuration. The information signal block $\mathbf{S}\in {\mathbb{C}}^{K\times N_f}$ at the input of the BS transceiver for the user k is given as

$${\mathbf{s}}_k=[s_1,s_2,\dots ,s_{N_f}],\forall k,$$

(1)

and for the subchannel n,

$${\mathbf{s}}_n={[s_1,s_2,\dots ,s_K]}^T,\forall n,$$

(2)

where $N_f$ and $N_s$ are the number of subchannels and the number of symbols per subchannel, respectively. In a subchannel n, the information symbol vector is $\mathbf{s}\in {\mathbb{C}}^{N_s\times 1}$. We assume $N_s=K$, such that the transmit signal per subchannel n satisfying $\mathbb{E}\left\{{\mathbf{s}}_n{\mathbf{s}}_n^H\right\}=\frac{P_n}{K}{\mathbf{I}}_K$, where $P_n=P_T/N_f$ is the transmit power per subchannel and $P_T$ is the total transmit power of the BS. The transmit signal vector $\mathbf{X}$ is obtained from ${\mathbf{F}}^B\mathbf{S}$, where ${\mathbf{F}}^B\in {\mathbb{C}}^{N_t\times N_s}$ is the precoding matrix. The hybrid beamforming divides the precoding matrix into baseband digital precoding matrix ${\mathbf{F}}^{DB}\in {\mathbb{C}}^{N_{RF}\times N_s}$ and RF analog precoding matrix ${\mathbf{F}}^{AB}\in {\mathbb{C}}^{N_t\times N_{RF}}$, where $N_{RF}$ is the number of RF chains as shown in Figure 3. The transmit signal $\mathbf{X}\in {\mathbb{C}}^{N_t\times N_f}$ is given by

$$\begin{array}{ccc}\hfill \mathbf{X}& =& {\mathbf{F}}^B\mathbf{S}\hfill \\ & =& {\mathbf{F}}^{AB}{\mathbf{F}}^{DB}\mathbf{S}\hfill \end{array}$$

(3)

Also, the precoding matrix must satisfy

$$\begin{array}{ccc}\hfill \mathbb{E}\left\{tr\left(\mathbf{X}{\mathbf{X}}^H\right)\right\}& \le & P_T\hfill \\ \hfill \mathbb{E}\left\{tr\left({\mathbf{F}}^{AB}{\mathbf{F}}^{DB}\mathbf{S}{\mathbf{S}}^H{{\mathbf{F}}^{DB}}^H{{\mathbf{F}}^{AB}}^H\right)\right\}& \le & P_T\hfill \\ \hfill tr\left({\mathbf{F}}^{AB}{\mathbf{F}}^{DB}\mathbb{E}\left\{\mathbf{S}{\mathbf{S}}^H\right\}{{\mathbf{F}}^{DB}}^H{{\mathbf{F}}^{AB}}^H\right)& \le & P_T\hfill \end{array}$$

(4)

since $\mathbb{E}\left\{\mathbf{S}{\mathbf{S}}^H\right\}=\frac{P_T}{N_{f}K}{\mathbf{I}}_{K{N}_f}$, therefore,

$$tr\left({\mathbf{F}}^{AB}{\mathbf{F}}^{DB}{{\mathbf{F}}^{DB}}^H{{\mathbf{F}}^{AB}}^H\right)\le N_{f}K$$

(5)

The transmit signal in subchannel n is ${\mathbf{x}}_n\in {\mathbb{C}}^{N_t\times 1}$. Thus, the received signal vector ${\mathbf{y}}_n\in {\mathbb{C}}^{K\times 1}$ at K users in subchannel n is given by

$$\begin{array}{ccc}\hfill {\mathbf{y}}_n& =& {\mathbf{H}}_n^H{\mathbf{x}}_n+{\mathbf{w}}_n\hfill \\ & =& {\mathbf{H}}_n^H{\mathbf{F}}^{AB}{\mathbf{F}}_n^{DB}{\mathbf{s}}_n+{\mathbf{w}}_n\hfill \end{array}$$

(6)

where ${\mathbf{H}}_n\triangleq [{\mathbf{h}}_{1,n},\dots ,{\mathbf{h}}_{K,n}]\in {\mathbb{C}}^{N_t\times K}$ is the channel matrix with ${\mathbf{h}}_{k,n}={[h_{1,k},\dots ,h_{N_t,k}]}^H$ being the channel vector from BS to user k in subchannel n, ${\mathbf{x}}_n={\mathbf{F}}_n^{DB}{\mathbf{F}}^{AB}{\mathbf{s}}_n$, and ${\mathbf{w}}_n\sim \mathcal{CN}(\mathbf{0},{\sigma }^2{\mathbf{I}}_K)$ be the additive white Gaussian noise (AWGN) in subchannel n at the users. The RF beamforming ${\mathbf{F}}^{RF}$ is performed in time domain and the same beamforming is applied on all subchannels, whereas, the digital beamforming ${\mathbf{F}}_n^{DB}$ is performed in frequency domain on the per subchannel basis [11]. In the $n^{th}$ subchannel, the $j^{th}$ UE receives the sum of all transmitted signals for K UEs over its MIMO channel ${\mathbf{H}}_{j,i}$ as

$$y_{j,n}=\sum _{k=1}^K{\mathbf{h}}_{j,n}^H{\mathbf{x}}_{k,n}+w_{j,n}$$

(7)

where ${\mathbf{h}}_{j,n}$ is the $N_t\times 1$ channel vector. We denote the rank of the channel matrix ${\mathbf{H}}_{j,n}$ by $r_{j,n}$, where $0\le r_{j,n}\le min(K,N_t),\forall n$. In matrix form, the above equation is given as

$${\mathbf{y}}_{j,n}={\mathbf{h}}_{j,n}^H{\mathbf{x}}_n+{\mathbf{w}}_{j,n}$$

(8)

The $1\times N_f$ received signal at the $k^{th}$ UE is given by

$${\mathbf{y}}_k=[{\mathbf{h}}_{k,1}^H{\mathbf{F}}^{AB}{\mathbf{F}}_1^{DB}{\mathbf{s}}_1,\dots ,{\mathbf{h}}_{k,N_f}^H{\mathbf{F}}^{AB}{\mathbf{F}}_{N_f}^{DB}{\mathbf{s}}_{N_f}]+{\mathbf{w}}_k,$$

(9)

Combining the signals for all UEs in a K dimensional received signal vector $\mathbf{y}={[{\mathbf{y}}_1,\dots ,{\mathbf{y}}_K]}^H$, we get the system equation as

$$\mathbf{Y}={\mathbf{H}}^H{\mathbf{F}}^{AB}{\mathbf{F}}^{DB}\mathbf{S}+\mathbf{W},$$

(10)

where $\mathbf{Y},\mathbf{W}\in {\mathbb{C}}^{K\times N_f}$.

2.1. Channel Model

Generally, massive MIMO channel models are categorized in two types (i) analytical models and (ii) physical models [4]. Analytical models are commonly used for the theoretical analysis of wireless communication systems. The most commonly used analytical model is Kronecker channel model. It is a correlation-based model and characterizes the MIMO channel matrix in terms of the separate transmit and receive side spatial correlation matrices [24],

$$\begin{array}{cc}\hfill {\mathbf{R}}_H& =\mathbb{E}\left\{vec\left\{\mathbf{H}\right\}vec\left\{{\mathbf{H}}^H\right\}\right\}\hfill \\ & ={\mathbf{R}}_{tx}\otimes {\mathbf{R}}_{rx}\hfill \end{array}$$

(11)

under the above assumptions, the channel model $\mathbf{H}$ is simplified to Kronecker model,

$$\mathbf{H}={\mathbf{R}}_{rx}^{1/2}\mathbf{K}{\mathbf{R}}_{tx}^{1/2}$$

(12)

where $\mathbf{K}\sim \mathcal{CN}(0,1)$ is an i.i.d. unit variance MIMO channel matrix, ${\mathbf{R}}_{tx}$ and ${\mathbf{R}}_{rx}$ are the transmit and receive corrrelation matrices, respectively. The transmit and receive correlation matrices are given as [24],

$${\mathbf{R}}_{tx}=\mathbb{E}\left\{{\mathbf{H}}^H\mathbf{H}\right\},{\mathbf{R}}_{rx}=\mathbb{E}\left\{\mathbf{H}{\mathbf{H}}^H\right\}$$

(13)

The physical models explicitly model wave propagation parameters like the complex amplitude, DoD, DoA, and delay of an MPC [24,25]. MmWave propagation leads to limited spatial scattering due to the high free-space pathloss. In addition, the large tightly packed antenna arrays lead to high levels of antenna correlation. The sparse scattering and antennas spatial correlation makes many of the commonly used statistical fading distributions inaccurate for mmWave channel modeling. Therefore, we use extended Saleh-Valenzuela model, which accurately describes the mathematical structure present in mmWave channels [26,27]. For simplicity, we assume that each scattering cluster around the transmitter and receiver contributes a single propagation path [28].

In general, the mmWave MIMO channel matrix between the BS with $N_t$ transmit antennas and a user k with $n_r$ receive antennas in subchannel n, can be modeled as double directional channel,

$${\mathbf{H}}_{k,n}=\sqrt{\frac{N_t{n}_r}{{\rho }_{k,n}L}}\sum _{l=0}^L{\alpha }_{k,n,l}\mathbf{a}\left({\varphi }_{k,n,l}\right){\mathbf{b}}^H\left({\theta }_{k,n,l}\right),$$

(14)

where L is the total number of multipaths, ${\alpha }_{k,n,l}$ is the complex gain of the $l^{th}$ path with i.i.d. $\mathcal{C}\mathcal{N}(0,1)$, and ${\rho }_{k,n}$ is the distance dependent pathloss between the BS and user k[29]. The LOS path is included with $l=0$. Moreover, $\mathbf{a}$ and $\mathbf{b}$ are the receive and transmit steering vectors, respectively. The variables ${\varphi }_{k,n,l}\in [0,2\pi )$ and ${\theta }_{k,n,l}\in [0,2\pi )$ are the $l^{th}$ path’s azimuth angles (boresight angles in the receive array and transmit array) of arrival and departure, respectively. The steering vectors are given by

$$\mathbf{a}\left({\varphi }_{k,n,l}\right)=\frac{1}{\sqrt{N_r}}{[a_1\left({\varphi }_l\right),\dots ,a_{n_r}\left({\varphi }_{k,n,l}\right)]}^T$$

(15)

$$\mathbf{b}\left({\theta }_{k,n,l}\right)=\frac{1}{\sqrt{N_t}}{[b_1\left({\theta }_l\right),\dots ,b_{N_t}\left({\theta }_{k,n,l}\right)]}^T$$

(16)

The elements of transmit and receive steering vectors are given by

$$b_i\left({\theta }_{k,n,l}\right)=e^{-j\omega {\tau }_{i,l}}=e^{-j2\pi (i-1)\frac{d_t}{\lambda }sin\left({\theta }_{k,n,l}\right)},i=1,2,\dots ,N_t$$

(17)

$$a_i\left({\varphi }_{k,n,l}\right)=e^{-j\omega {\tau }_{i,l}}=e^{-j2\pi (i-1)\frac{d_r}{\lambda }sin\left({\varphi }_{k,n,l}\right)},i=1,2,\dots ,n_r$$

(18)

where $\lambda$ is the wavelength, $\omega =\frac{2\pi }{\lambda }$, ${\tau }_i$ is the beamforming delay, and $d_t$ and $d_r$ are the antenna spacing at the transmitter and receiver, respectively.

The channel matrix in (14) can also be written in more compact form as

$${\mathbf{H}}_{k,n}=\sqrt{\nu }{\mathbf{A}}_{k,n}{\mathbf{D}}_{k,n}{\mathbf{B}}_{k,n}^H$$

(19)

where $\nu =\frac{N_t{n}_r}{\rho L}$ and, ${\mathbf{A}}_{k,n}$ and ${\mathbf{B}}_{k,n}$ consist of stacked steering vectors of AoA and AoD, respectively, i.e., ${\mathbf{A}}_{k,n}=[\mathbf{a}\left({\varphi }_{k,n,1}\right),\mathbf{a}\left({\varphi }_{k,n,2}\right),\dots ,\mathbf{a}\left({\varphi }_{k,n,L}\right)]$ and ${\mathbf{B}}_{k,n}=[\mathbf{b}\left({\theta }_{k,n,1}\right),\mathbf{b}\left({\theta }_{k,n,2}\right),\dots ,\mathbf{b}\left({\theta }_{k,n,L}\right)]$. The matrix ${\mathbf{D}}_{k,n}$ is a diagonal matrix, given as ${\mathbf{D}}_{k,n}=diag\{{\alpha }_{k,n,1},{\alpha }_{k,n,2},\dots ,{\alpha }_{k,n,L}\}$. The small scale fading at user k in subchannel n in multipath component (MPC) is given by ${\alpha }_{k,n,l}$ with zero mean and variance ${\sigma }_{k,n,l}^2$. Assume that each MPC is i.i.d. such that ${\sum }_{l=1}^L{\sigma }_{k,n,l}^2=1$. We can express the channel model in (19) as

$${\mathbf{H}}_{k,n}=\sqrt{\nu }{\mathbf{A}}_{k,n}{\mathsf{\Sigma }}_{k,n}{\overline{\mathbf{D}}}_{k,n}{\mathbf{B}}_{k,n}^H,$$

(20)

where ${\mathsf{\Sigma }}_{k,n}=diag\{{\sigma }_{k,n,1}^2,{\sigma }_{k,n,2}^2\dots ,{\sigma }_{k,n,L}^2\}$ and ${\overline{\mathbf{D}}}_{k,n}=diag\{{\overline{\alpha }}_{k,n,1},{\overline{\alpha }}_{k,n,2}\dots ,{\overline{\alpha }}_{k,n,L}\}$ with ${\overline{\alpha }}_{k,n,l}=\frac{{\alpha }_{k,n,l}}{{\sigma }_{k,n,l}}$ such that $E\left\{{\overline{\alpha }}_{k,n,l}\right\}=0$ and $E\left\{{\overline{\alpha }}_{k,n,l}^2\right\}=1$.

Substituting (20) in (13) and averaging over small scale fading, we get the transmit and receive correlation matrices for user k in the subchannel n as

$${\mathbf{R}}_{tx,k,n}=\nu {\mathbf{B}}_{k,n}{\mathsf{\Sigma }}^2{\mathbf{B}}_{k,n}^H,$$

(21)

$${\mathbf{R}}_{rx,k,n}=\nu {\mathbf{A}}_{k,n}{\mathsf{\Sigma }}^2{\mathbf{A}}_{k,n}^H,$$

(22)

For mmWave massive MIMO systems with large number of antennas, the steering vectors are asymptotically orthogonal to each other [6]:

$${\mathbf{a}}^H\left({\varphi }_{k,n}\right)\mathbf{a}\left({\varphi }_{k,n^{\prime }}\right)\approx 0,$$

$${\mathbf{b}}^H\left({\theta }_{k,n}\right)\mathbf{b}\left({\theta }_{k,n^{\prime }}\right)\approx 0.$$

(23)

Moreover, in mmWave massive MIMO, acquisition of the instantaneous full CSI is not practical. Instead, an average CSI in terms of $\left[{\mathbf{A}}_{k,n}\right]$, $\left[{\mathbf{B}}_{k,n}\right]$, and $\left[{\mathsf{\Sigma }}_{k,n}\right]$ is a practical solution for the beamforming design because the coherence time of the channel statistics based CSI is of the order of few seconds or more as compared to the small scale of the order of milli-second [6].

2.2. Problem Formulation

The hybrid beamforming divides the beamforming matrix into two parts: covariance-based pre-beamforming matrix ${\mathbf{F}}^{AB}$ realized by analog beamformers and the reduced dimension MU-MIMO digital precoding based on the effective channel ${\mathbf{H}}^H{\mathbf{F}}^{AB}$ (omitting the subchannel subscript for simplicity). We assume that K users are divided into G groups, such that, the group g contains $K_g$ number of users. Since users are near the ground level and surrounded by the scatterers compared to the scatterer-free elevated base-station, we assume one-ring model [1] and all users in group g experience the same azimuth center angle (${\theta }_g$) and angular spread (${\Delta }_g$). In this case, ${\mathbf{R}}_{rx}=\mathbf{I}$ in (12), therefore, the channel covariance matrix of each user in group g is given by [30]

$${\mathbf{R}}_g=\mathbb{E}\left\{{\mathbf{h}}_{g_k}{\mathbf{h}}_{g_k}^H\right\}$$

(24)

for which the eigenvalue decomposition gives

$${\mathbf{R}}_g={\mathbf{U}}_g{\mathsf{\Lambda }}_g{\mathbf{U}}_g^H$$

(25)

where ${\mathbf{U}}_g\in {\mathbb{C}}^{N_t\times r_g}$ is a tall unitary matrix (${\mathbf{U}}_g{\mathbf{U}}_g^H=\mathbf{I}$) comprises the eigenvectors of ${\mathbf{R}}_g$ and ${\mathsf{\Lambda }}_g\in {\mathbb{R}}^{r_g\times r_g}$ is diagonal matrix with $r_g$ nonzeros positive eigenvalues along the diagonal. The $i,j-th$ element of covariance matrix ${\mathbf{R}}_g$ represents the correlation between the channel coefficients antenna element i and j as

$${\left[{\mathbf{R}}_g\right]}_{i,j}=\frac{1}{2{\Delta }_g}{\int }_{{\theta }_g-{\Delta }_g}^{{\theta }_g+{\Delta }_g}{e}^{-j2\pi \frac{d}{\lambda }(i-j)sin\theta }d\theta ,$$

(26)

where d is the distance between antenna elements of ULA and $\lambda$ is the wavelength of carrier frequency. Using the Karhunen-Loeve representation, the channel vector of user k in group g is given as

$${\mathbf{h}}_{g_k}={\mathbf{U}}_g{\mathsf{\Lambda }}_g^{1/2}{\mathbf{z}}_g={\mathbf{U}}_g{\tilde{\mathbf{h}}}_{g_k}$$

(27)

where ${\mathbf{z}}_g\in {\mathbb{C}}^{N_t\times 1}\sim \mathcal{CN}(\mathbf{0},{\mathbf{I}}_{r_g})$ and ${\tilde{\mathbf{h}}}_{g_k}$ is beam domain channel. For large $N_t$, ${\mathbf{U}}_g$ tends to discrete Fourier transform (DFT) matrix ${\Delta }_{N_t}\in {\mathbb{C}}^{N_t\times N_t}$ [31]. Each column of ${\mathbf{U}}_g$ represents one direction of angle-of-departure (AoD), i.e., a beam.

Alternatively, for the case, when dominant eigenvalues ${\widehat{r}}_g\le r_g$, then, the channel matrix can be written as ([13], Equation (5))

$${\mathbf{h}}_{g_k}=N_t^{1/2}{\mathbf{R}}_g^{1/2}{\mathbf{z}}_{g_k}$$

(28)

The limited feedback-based hybrid beamforming consists of analog pre-beamforming matrix ${\mathbf{F}}_g^{AB}\in {\mathbb{C}}^{N_t\times N_{RF,g}}$ responsible for spatial group formation and inter-group interference mitigation; and the digital multi-users precoding matrix ${\mathbf{F}}_g^{DB}\in {\mathbb{C}}^{N_{RF,g}\times S_g}$ for spatial multiplexing inside the group and inter-user interference mitigation. Here, $N_{RF,g}$ is the number of RF chains for group g such that $S_g<N_{RF,g}<{\widehat{r}}_g$ and $S_g$ is the number of multi-carrier information symbols vectors for group g with $N_{RF}={\sum }_{g=1}^G{N}_{RF,g}$ and $S={\sum }_{g=1}^G{S}_g$. The overall analog pre-beamforming matrix ${\mathbf{F}}^{AB}\in {\mathbb{C}}^{N_t\times N_{RF}}$ is given by

$${\mathbf{F}}^{AB}=[{\mathbf{F}}_1^{AB},\dots ,{\mathbf{F}}_G^{AB}],$$

(29)

and the overall digital beamforming matrix ${\mathbf{F}}^{DB}\in {\mathbb{C}}^{N_{RF}\times N_s}$ is given by

$${\mathbf{F}}^{DB}=diag[{\mathbf{F}}_1^{DB},\dots ,{\mathbf{F}}_G^{DB}]$$

(30)

and the overall channel matrix

$$\mathbf{H}=[{\mathbf{H}}_1,\dots ,{\mathbf{H}}_G],$$

(31)

where the channel matrix of group g is defined as ${\mathbf{H}}_g\triangleq [{\mathbf{h}}_{g_1},\dots ,{\mathbf{h}}_{g_{K_g}}]$.

The analog pre-beamforming ${\mathbf{F}}_g^{AB}$ is based on the slowly varying channel covariance matrix ${\mathbf{R}}_g$ and can be implemented by the DFT matrix (when $N_t$ is large), whereas, the digital beamformer ${\mathbf{F}}_g^{DB}$ is based on the instantaneous channel information of the reduced dimension effective channel ${\mathbf{H}}_g^H{\mathbf{F}}_g^{AB}$. The overall effective channel is given by

$${\mathbf{H}}_{eff}^H={\mathbf{H}}^H{\mathbf{F}}^{AB}=\left[\begin{array}{cccc}{\mathbf{H}}_1^H{\mathbf{F}}_1^{AB}& {\mathbf{H}}_1^H{\mathbf{F}}_2^{AB}& \cdots & {\mathbf{H}}_1^H{\mathbf{F}}_G^{AB}\\ {\mathbf{H}}_2^H{\mathbf{F}}_1^{AB}& {\mathbf{H}}_2^H{\mathbf{F}}_2^{AB}& \cdots & {\mathbf{H}}_2^H{\mathbf{F}}_G^{AB}\\ ⋮& ⋮& \ddots & ⋮\\ {\mathbf{H}}_G^H{\mathbf{F}}_1^{AB}& {\mathbf{H}}_G^H{\mathbf{F}}_2^{AB}& \cdots & {\mathbf{H}}_G^H{\mathbf{F}}_G^{AB}\end{array}\right]$$

(32)

The excessive pilot transmission in downlink and feedback in uplink of FDD system can be reduced by only sending the group-wise average CSI based channel estimates in uplink. This is accomplished by using the diagonal elements ${\mathbf{H}}_g^H{\mathbf{F}}_g^{AB}$ as feedback information with the size of $K_g\times N_{RF,g}$ for $g=1,\dots G$. The analog pre-beamforming is designed in such a way that the other elements of matrix (32) ${\mathbf{H}}_g^H{\mathbf{F}}_{g^{\prime }}^{AB}\approx 0$ for all $g^{\prime }\ne g$. This group-wise division creates virtual sectors, each group corresponds to a virtual sector [30].

The second order channel statistics-based RF beamformer ${\mathbf{F}}^{AB}$ remains the same across multiple coherence blocks which gives the effective instantaneous channel between BS and user k as

$${\mathbf{h}}_{n,g_k,eff}^H\triangleq {\mathbf{h}}_{n,g_k}^H{\mathbf{F}}_g^{AB},$$

(33)

with ${\mathbf{h}}_{n,g_k,eff}\in {\mathbb{C}}^{N_{RF,g}\times 1}$. Therefore, channel statistics-based CSI sufficiently reduces the feedback overhead on each user, otherwise, for instantaneous CSI, each user have to send the $N_t\times 1$ size of channel estimate on the uplink channel. The covariance of effective channel ${\mathbf{h}}_{n,g_k,eff}^H$ is given by using (13) as,

$$\begin{array}{ccc}\hfill \mathbb{E}\left\{{\mathbf{h}}_{n,g_k,eff}{\mathbf{h}}_{n,g_k,eff}^H\right\}& =& \mathbb{E}\left\{{{\mathbf{F}}_g^{AB}}^H{\mathbf{h}}_{n,g_k}{\mathbf{h}}_{n,g_k}^H{\mathbf{F}}_g^{AB}\right\}\hfill \\ & =& \nu {{\mathbf{F}}_g^{AB}}^H{\mathbf{B}}_{k,n}{\mathsf{\Sigma }}^2{\mathbf{B}}_{k,n}^H{\mathbf{F}}_g^{AB}\hfill \end{array}$$

(34)

The analog beamformer consists of columns of the DFT matrix, which can be easily implemented by phase shifter network. Therefore, ${\mathbf{F}}_{n,g}^{AB}$ can be obtained by eigenvalue decomposition of channel covariance matrix. With the group-wise hybrid beamforming, the received signal ${\mathbf{y}}_{g,n}$ for group g in subchannel n becomes

$${\mathbf{y}}_{g,n}={\mathbf{H}}_{g,n}^H{\mathbf{F}}_g^{AB}{\mathbf{F}}_{g,n}^{DB}{\mathbf{s}}_{g,n}+{\mathbf{H}}_{g,n}^H\sum _{g^{\prime }\ne g}{\mathbf{F}}_{g^{\prime }}^{AB}{\mathbf{F}}_{g^{\prime },n}^{DB}{\mathbf{s}}_{g^{\prime },n}+{\mathbf{w}}_{g,n}$$

(35)

and the received signal of user k in group g in subchannel n is given by

$$\begin{array}{cc}\hfill y_{g_k,n}& ={\mathbf{h}}_{g_k,n}^H{\mathbf{F}}_g^{AB}{\mathbf{f}}_{g_k,n}^{DB}{\mathbf{s}}_{g_k,n}+\underset{\mathrm{Inter}-\mathrm{user}\mathrm{interference}}{\underbrace{{\mathbf{h}}_{g_k,n}^H\sum _{k^{\prime }\ne k}{\mathbf{F}}_g^{AB}{\mathbf{f}}_{g_{k^{\prime }},n}^{DB}{\mathbf{s}}_{g_{k^{\prime }},n}}}\hfill \\ & +\underset{\mathrm{Inter}-\mathrm{group}\mathrm{interference}}{\underbrace{{\mathbf{h}}_{g_k,n}^H\sum _{g^{\prime }\ne g}\sum _j{\mathbf{F}}_{g^{\prime }}^{AB}{\mathbf{f}}_{g_j^{\prime },n}^{DB}{\mathbf{s}}_{g_j^{\prime },n}}}+w_{g_k,n}\hfill \end{array}$$

(36)

The received signal to interference and noise ratio (SINR) at the user k in group g and subchannel n is given by

$$SIN{R}_{g_k,n}=\frac{|{\mathbf{h}}_{g_k,n}^H{\mathbf{F}}_g^{AB}{\mathbf{f}}_{g_k,n}^{DB}{|}^2}{|{\mathbf{h}}_{g_k,n}^H{|}^2\left({\sum }_{k^{\prime }\ne k}|{\mathbf{F}}_g^{AB}{\mathbf{f}}_{g_{k^{\prime }},n}^{DB}{|}^2+{\sum }_{g^{\prime }\ne g}{\sum }_j{\left|{\mathbf{F}}_{g^{\prime }}^{AB}{\mathbf{f}}_{g_j^{\prime },n}^{DB}\right|}^2\right)+{\sigma }^2}$$

(37)

The spectral efficiency of user k in group g and subchannel n is expressed as

$$R_{g_k,n}={\mathsf{\Psi }}_{g_k,n}{log}_2(1+SIN{R}_{g_k,n}),$$

(38)

where ${\mathsf{\Psi }}_{g_k,n}$ is the binary variable such that it is equal to 1 if user k is selected in group g in the subchannel n. In order to achieve balance tradeoff between throughput and fairness [32], we use proportional fairness (PF) based throughput maximization. We define per user proportional fairness metric as

$$U({\mathbf{F}}_g^{AB},{\mathbf{f}}_{g_k,n}^{DB})=\frac{R_{g_k,n}\left(t\right)}{{\overline{R}}_{g_k,n}\left(t\right)},\forall n,g_k,$$

(39)

where ${\overline{R}}_{g_k,n}\left(t\right)$ is average throughput (moving average) over a past window of length $T_w=1/\alpha$ [33], as

$${\overline{R}}_{g_k,n}\left(t\right)=\alpha R_{g_k,n}(t-1)+(1-\alpha ){\overline{R}}_{g_k,n}(t-1),$$

(40)

The large number of antennas in massive MIMO systems enable the use of the eigenmodes of the channel covariance matrix, i.e., ${\mathbf{B}}_{k,n}$ comprises of the columns of the DFT matrix [6]. DFT-based beams with $N_t=16$ and $N_t=64$ are shown in Figure 4a,b, respectively.

The beam steering matrix ${\mathbf{B}}_{k,n}$ consists of selected columns of $N_t\times N_t$ DFT matrix ${\Delta }_{N_t}$ such that

$${\mathbf{F}}_g^{AB}={\mathbf{B}}_{k,n}={\Delta }_{N_t}{\mathbf{Y}}_n$$

(41)

where ${\Delta }_{N_t}=[{\mathbf{b}}_1,{\mathbf{b}}_2,\dots ,{\mathbf{b}}_{N_t}]$ consisting of all eigenmodes and ${\mathbf{Y}}_n$ is an $N_t\times r_R$ binary beam selection matrix, with $r_R$ is the rank of the channel covariance matrix. The selection matrix ${\mathbf{Y}}_n\in {\mathbb{C}}^{N_t\times r_R}$ with only a single one on each row and column such that ${\sum }_i{\left[{\mathbf{Y}}_n\right]}_{i,j}=1\forall j$. Now we formulate our optimization problem for joint spatio–radio resource allocation and precoders design with the objective to maximize the utility function as

$$\begin{array}{cc}\hfill \underset{{\mathbf{F}}_g^{AB},{\mathbf{f}}_{g_k,n}^{DB}}{max}& \sum _{g=1}^G\sum _{n=1}^{N_f}\sum _{k=1}^{K_g}U\left({\mathbf{F}}_g^{AB},{\mathbf{f}}_{g_k,n}^{DB}\right)\hfill \\ \hfill \mathrm{subject}\mathrm{to}& \\ \hfill C1:& tr\left({{\mathbf{F}}_n^{DB}}^H{{\mathbf{F}}^{AB}}^H{\mathbf{F}}^{AB}{\mathbf{F}}_n^{DB}\right)\le P_n,\forall n\hfill \\ \hfill C2:& rank\left({\mathbf{F}}^{AB}{\mathbf{F}}^{DB}\right)\le N_{RF},\hfill \\ \hfill C3:& \sum _{k=1}^{K_g}{\mathsf{\Psi }}_{k,n}\le K_g,\forall g,n,\hfill \\ \hfill C4:& tr\left({\mathbf{Y}}_n\right)\le G,\forall n,\hfill \\ \hfill C5:& \left[{\mathsf{\Psi }}_{k,n}\right],\left[{\mathbf{Y}}_n\right]\in \{0,1\}.\hfill \end{array}$$

(42)

The above optimization problem is a mixed integer programming (MIP) problem with coupling between the digital and RF precoders in the power constraint. This MIP problem is NP-hard [14].

3. Relaxed-Convex Transformation

Though the above MIP optimization problem is NP-hard, it can be transformed to a relaxed convex optimization problem by (i) relaxing the binary integer constraints to real number between 0 and 1 [14], and (ii) decoupling the digital and analog precoders. For decoupling purpose, we make use of change of variables ${\mathbf{F}}_n^{DB}={\left({{\mathbf{F}}^{AB}}^H{\mathbf{F}}^{AB}\right)}^{-\frac{1}{2}}{\tilde{\mathbf{F}}}_n^{DB}$, where ${\tilde{\mathbf{F}}}_n^{DB}$ is the equivalent digital precoder [34]. Thus, the problem in (42) can be written as

$$\begin{array}{cc}\hfill \underset{{\mathbf{F}}_g^{AB},{\tilde{\mathbf{f}}}_{g_k,n}^{DB}}{max}& \sum _{g=1}^G\sum _{n=1}^{N_f}\sum _{k=1}^{K_g}{\tilde{U}}_{g_k,n}\left({\mathbf{F}}_g^{AB},{\mathbf{f}}_{g_k,n}^{DB}\right)\hfill \\ \hfill \mathrm{subject}\mathrm{to}\\ \hfill C1:& tr\left({{\tilde{\mathbf{F}}}_n^{DB}}^H{\tilde{\mathbf{F}}}_n^{DB}\right)\le P_n,\forall n\hfill \\ \hfill C2:& rank\left({\tilde{\mathbf{F}}}^{DB}\right)\le N_{RF},\hfill \\ \hfill C3:& \sum _{k=1}^{K_g}{\mathsf{\Psi }}_{k,n}\le K_g,\forall g,n,\hfill \\ \hfill C4:& tr\left({\mathbf{Y}}_n\right)\le G,\forall n,\hfill \\ \hfill C5:& 0\le \left[{\mathsf{\Psi }}_{k,n}\right]\le 1,0\le \left[{\mathbf{Y}}_n\right]\le 1,\hfill \end{array}$$

(43)

For a given RF precoder ${\mathbf{F}}^{AB}$ and the knowledge of perfect CSI at the base-station, the digital precoder can be obtained by conventional MU-MIMO techniques, e.g., the zero-forcing and block diagonalization [15].

For the digital precoder, we adopt the ZF precoder for no multiuser interference among the users in each groups. The beamforming vector of user k is chosen to be orthogonal to the effective channel vectors of all the other users in the group. Zero-forcing is a suboptimal but low complexity approach within the linear precoders’ class. ZF precoder is asymptotically optimal among all downlink beamforming techniques in high SNR region. It guarantees high spectral efficiency for large-scale antennas with low-complexity linear processing [35]. For $N_t\gg N_r$, it has shown that zero-forcing beamforming can achieve up to $98\%$ of the non-linear dirty paper coding (DPC) capacity [36]. In order to make this paper self-contained, we describe the block diagonalization briefly. Since digital precoder is used to mitigate the multiuser interference within a groups and all groups are independent, we omit the subscript g. First we consider the downlink transmission over one subchannel n with the general case of BS with $N_t$ antennas and $K_n$ users with $n_k$ antennas each, such that ${\sum }_{k=1}^K{n}_k=N_r$. The downlink channel on the subchannel n is expressed as $N_r\times N_t$ matrix,

$${\mathbf{H}}_{n,eff}={[{\mathbf{H}}_{1,n,eff}^T,\dots ,{\mathbf{H}}_{K_n,n,eff}^T]}^T$$

(44)

For user k, we define the following $(N_r-n_k)\times N_t$ channel matrix

$${\mathbf{H}}_{k,n,eff}^{\prime }={[{\mathbf{H}}_{1,n,eff}^T,\dots ,{\mathbf{H}}_{K-1,n,eff}^T,{\mathbf{H}}_{k+1,n,eff}^T,\dots ,{\mathbf{H}}_{K_n,n,eff}^T]}^T$$

(45)

Let the rank of ${\mathbf{H}}_{k,n,eff}^{\prime }$ be denoted by $r_{k,n}^{\prime }$, then the nullspace of ${\mathbf{H}}_{k,n,eff}^{\prime }$ has dimension $N_t-r_{k,n}^{\prime }\ge n_k$. Performing the SVD of each user’s channel matrix in subchannel n leads to the following

$${\mathbf{H}}_{k,n,eff}^{\prime }={\mathbf{U}}_{k,n}^{\prime }{\mathsf{\Sigma }}_{k,n}^{\prime }{\mathbf{V}}_{k,n}^{{}^{\prime }H}={\mathbf{U}}_{k,n}^{\prime }{\mathsf{\Sigma }}_{k,n}^{\prime }{\left[{\mathbf{V}}_{k,n}^{{}^{\prime }\left(1\right)}{\mathbf{V}}_{k,n}^{{}^{\prime }\left(0\right)}\right]}^H,$$

(46)

where ${\mathbf{U}}_{k,n}^{\prime }$ and ${\mathbf{V}}_{k,n}^{\prime }$ are the unitary matrices. The columns of ${\mathbf{U}}_{k,n}^{\prime }$ are the left singular vectors of ${\mathbf{H}}_{k,n,eff}^{\prime }$, the columns of ${\mathbf{V}}_{k,n}^{\prime }$ are the right singular vectors of ${\mathbf{H}}_{k,n,eff}^{\prime }$, and ${\mathsf{\Sigma }}_{k,n}^{\prime }$ is a diagonal matrix in which the diagonal entries are the singular values of ${\mathbf{H}}_{k,n,eff}^{\prime }$. In the last equality of (46), ${\mathbf{V}}_{k,n}^{{}^{\prime }\left(1\right)}$ holds the first $r_{k,n}^{{}^{\prime }}$ right singular vectors of ${\mathbf{H}}_{k,n,eff}^{\prime }$ and ${\mathbf{V}}_{k,n}^{{}^{\prime }\left(0\right)}$ contains the $N_t-r_{k,n}^{{}^{\prime }}$ singular vectors of ${\mathbf{H}}_{k,n,eff}^{\prime }$ which are in the nullspace of ${\mathbf{H}}_{k,n,eff}^{\prime }$. The columns of ${\mathbf{V}}_{k,n}^{{}^{\prime }\left(0\right)}$ are best suited for user k beamforming matrix ${\mathbf{F}}_{k,n}^{DB}$, because they will provide zero interference at other UEs. Usually ${\mathbf{V}}_{k,n}^{{}^{\prime }\left(0\right)}$ contains more number of columns than the $n_k$, therefore we use some linear combinations of the columns of ${\mathbf{V}}_{k,n}^{{}^{\prime }\left(0\right)}$ to make at most $n_k$ columns.

$${\mathbf{H}}_{k,n,eff}{\mathbf{V}}_{k,n}^{{}^{\prime }\left(0\right)}={\mathbf{U}}_{k,n}\left[\begin{array}{cc}{\mathsf{\Sigma }}_{k,n}& \mathbf{0}\\ \mathbf{0}& \mathbf{0}\end{array}\right]{\left[\begin{array}{cc}{\mathbf{V}}_{k,n}^{\left(1\right)}& {\mathbf{V}}_{k,n}^{\left(0\right)}\end{array}\right]}^H,$$

(47)

where ${\mathbf{H}}_{k,n,eff}{\mathbf{V}}_{k,n}^{{}^{\prime }\left(0\right)}$ gives the matrix with columns as the linear combinations of the columns of ${\mathbf{V}}_{k,n}^{{}^{\prime }\left(0\right)}$. The right hand side of the equation is the SVD of ${\mathbf{H}}_{k,n,eff}{\mathbf{V}}_{k,n}^{{}^{\prime }\left(0\right)}$, where ${\mathsf{\Sigma }}_{k,n}$ is $r_{k,n}\times r_{k,n}$ diagonal matrix and ${\mathbf{V}}_{k,n}^{\left(1\right)}$ represents the $r_{k,n}$ singular vectors with nonzero singular values of ${\mathbf{H}}_{k,n,eff}{\mathbf{V}}_{k,n}^{{}^{\prime }\left(0\right)}$. The Equation (47) can also be written as,

$$\begin{array}{ccc}\hfill {\mathbf{H}}_{k,n,eff}{\mathbf{V}}_{k,n}^{{}^{\prime }\left(0\right)}& =& {\mathbf{U}}_{k,n}{\mathsf{\Sigma }}_{k,n}{{\mathbf{V}}_{k,n}^{\left(1\right)}}^H\hfill \\ \hfill {\mathbf{H}}_{k,n,eff}& =& {\mathbf{U}}_{k,n}{\mathsf{\Sigma }}_{k,n}{{\mathbf{V}}_{k,n}^{\left(1\right)}}^H{{\mathbf{V}}_{k,n}^{{}^{\prime }\left(0\right)}}^H\hfill \end{array}$$

(48)

The transmit beamforming matrix that maximizes the user k throughput without any inter-user interference is obtained as,

$${\mathbf{F}}_{k,n}^{DB}={\mathbf{V}}_{k,n}^{{}^{\prime }\left(0\right)}{\mathbf{V}}_{k,n}^{\left(1\right)}$$

(49)

The transmit digital beamforming matrix for subchannel n is defined as

$${\mathbf{F}}_n^{DB}=[{\mathbf{F}}_{1,n}^{DB},\dots ,{\mathbf{F}}_{K_n,n}^{DB}]{\mathbf{P}}_n^{1/2},$$

(50)

where ${{\mathbf{F}}_{k,n}^B}^H{\mathbf{F}}_{k,n}^B=\mathbf{I},1\le k\le K_n$ and ${\mathbf{P}}_n$ is a block diagonal matrix whose elements scale the power allocated to each interference-free virtual subchannel for all UEs. The receive combining matrix for this user is ${\mathbf{U}}_{k,n}$ [37].

In the case of single antenna users, complete diagonalization is achieved entirely at the BS by channel inversion, i.e., ${\mathbf{F}}_n^{DB}={\left({\mathbf{H}}_{n,eff}^H\right)}^{†}$, where ${\left({\mathbf{H}}_{n,eff}^H\right)}^{†}$ is the pseudo-inverse of ${\mathbf{H}}_{n,eff}^H$ [38].

$$\begin{array}{cc}\hfill {\mathbf{F}}_n^{DB}=& {\beta }_n{\left({\mathbf{H}}_{n,eff}^H\right)}^{†}\hfill \\ \hfill {\mathbf{f}}_{1,n}^{DB},\dots ,{\mathbf{f}}_{K,n}^{DB}]=& {\beta }_n{\left[{[{\mathbf{h}}_{1,n},\dots ,{\mathbf{h}}_{K,n}]}^H{\mathbf{F}}^{AB}\right]}^{†}\hfill \end{array}$$

(51)

where ${\beta }_n$ is a normalization factor chosen to satisfy the power constraint and is given by

$${\beta }_n^2=\frac{K}{\parallel {\mathbf{F}}^{AB}{\mathbf{F}}_n^{DB}{\parallel }_F^2}$$

(52)

Using the definition of the pseudo-inverse, we get,

$${\mathbf{F}}_n^{DB}={\beta }_n{\left({\mathbf{H}}_{n,eff}{\mathbf{H}}_{n,eff}^H+N_{RF}\varrho {\mathbf{I}}_{N_{RF}}\right)}^{-1}{\mathbf{H}}_{n,eff}$$

(53)

where $\varrho$ is the regularization parameter, $\varrho =0$ for ZF precoding and $\varrho =\frac{N_s}{N_{RF}\eta }$ for regularized ZF, with $\eta =P_{T,n}/{\sigma }^2$. Lastly, introducing the group subscript again, the SINR of user $g_k$ is given by

$$SIN{R}_{g_k,n}=\frac{\frac{P_{T,n}}{K_g}{\beta }^2{\left|{\mathbf{h}}_{g_k,n}^H{\mathbf{F}}_g^{AB}{\mathbf{F}}_{g,n}^{DB}{{\mathbf{F}}_{g,n}^{DB}}^H{{\mathbf{F}}_g^{AB}}^H{\mathbf{h}}_{g_k,n}\right|}^2}{|{\mathbf{h}}_{g_k,n}^H{|}^2\left({\sum }_{k^{\prime }\ne k}|{\mathbf{F}}_g^{AB}{\mathbf{f}}_{g_{k^{\prime }},n}^{DB}{|}^2+{\sum }_{g^{\prime }\ne g}{\sum }_j{\left|{\mathbf{F}}_{g^{\prime }}^{AB}{\mathbf{f}}_{g_j^{\prime },n}^{DB}\right|}^2\right)+{\sigma }^2}$$

(54)

and the PF sum rate is calculated as

$$\widehat{U}=\sum _{g=1}^G\sum _{n=1}^{N_f}\sum _{k=1}^{K_g}{\tilde{U}}_{g_k,n}.$$

(55)

4. Suboptimal Solution

Joint optimization of analog and digital beamformers is challenging because they use different channel information for the design of analog and digital beamformers. Hybrid beamforming methods consider decoupled designs of analog and digital beamforming to reduce the complexity of joint optimization, but the main challenge remains the use of different channel information. To approximate the optimal solution to this mixed integer programming problem, we summarize our proposed algorithm below:

The analog precoder is formed by selecting $K_g$ columns of DFT matrix of eigenvectors of channel covariance matrix ${\mathbf{R}}_g$ of users’ group g in (41) to minimize the inter-group interference ${\mathbf{I}}_g$,

$$\begin{array}{cc}\hfill \underset{\mathbf{Y}}{min}& {\mathbf{I}}_g\left(\mathbf{Y}\right)\hfill \\ \hfill \mathrm{subject}\mathrm{to}& \\ \hfill & tr\left({\mathbf{Y}}_n\right)\le G,\forall n,\hfill \end{array}$$

(56)

where ${\mathbf{I}}_g={\sum }_{i=1}^G{\sum }_{\begin{array}{c}j=1\\ j\ne i\end{array}}^G{\left|{\mathbf{H}}_i^H{\mathbf{F}}_j^{AB}\right|}^2$. To solve the MIP problem, we divide the solution into two parts. In the first part, we get the analog precoder using the selected columns of the DFT matrix which maximize the PF sum-rate. The inherent benefit of the DFT matrix is its constant modulus which enables the use of analog phase shifters and RF switches to realize the analog beamforming. In the second part, for the given analog precoder, intra-group users scheduling is performed and a ZF digital precoder is designed to maximize the sum-rate utility function. The decoupling of the analog and digital precoders design makes the solution suboptimal but tractable [34]. The joint hybrid beamforming and user scheduling Algorithm 1 takes $K_g$, $N_f$, $N_{RF}$, $N_t$, and K. It generates the analog beamforming matrix ${\mathbf{F}}^{AB}$, digital beamforming matrix ${\mathbf{F}}^{DB}$, and the set of users in each group. The first part of the algorithm (line 9 to line 19) forms DFT/eigenmodes-based analog beamforming using limited statistical CSI feedback from the users. Beam and users pairing within each group is taken place in this part of the algorithm. The while loop at line 12 executes till all the binary combinations in $N_t\times N_t$ are exhausted with the condition that each column contain exactly one binary 1 and total number of 1s are equal to the number of streams (or number of RF chains). The second part (starts from line 39) assigns the radio resources to users to maximize the utility function.

Algorithm 1 Joint Resource Allocation and Hybrid Beamforming Design Algorithm.

1: Inputs   2: $K_g$: Number of users per group   3: $N_f$: Number of subchannels   4: $K_n$: Number of UEs to be scheduled in subchannel n   5: $N_{RF}$: Number of RF chains   6: $N_t$: Number of transmit antennas   7: Initialization   8: ${\mathcal{K}}_g=\{1,\dots ,K_g\}$, $U{\left({\mathbf{F}}_n^{DB}\right)}^{last}=0$, ${\mathbf{Y}}_g^{last}=\mathbf{0}$, ${\mathcal{K}}_n=\left\{\right\},\forall n$ {Average CSI-based Beam Selection and RF Precoder Design}   9: Evaluate channel covariance matrix ${\mathbf{R}}_g$ from CSI at users: ${\mathbf{R}}_g=\mathbb{E}\left\{{\mathbf{h}}_{g_k}{\mathbf{h}}_{g_k}^H\right\}$ 10: Find transmit steering ${\mathbf{B}}_{k,n}$ by Eigenvector based unitary matrix ${\mathbf{U}}_g$ of ${\mathbf{R}}_g$ or DFT matrix ${\Delta }_{N_t}$. 11: while$g\le G$do 12:     while $tr\left({\mathbf{Y}}_g\right)\le N_{RF,g}$, set $\{{\left[{\mathbf{Y}}_g\right]}_{i,i},\dots ,{\left[{\mathbf{Y}}_g\right]}_{N_t,N_t}\}=\{0,1\}$ do 13:         Calculate $U\left({\mathbf{Y}}_g\right)$, employing zero forcing ${\mathbf{F}}_{n,g}^{DB}$ 14:         if $U\left({\mathbf{Y}}_g\right)\ge U\left({\mathbf{Y}}_g^{last}\right)$ then 15:              ${\mathbf{Y}}_g^{last}\leftarrow {\mathbf{Y}}_g$ 16:         end if 17:     end while 18: end while 19: Get ${\mathbf{F}}_g^{AB}={\Delta }_{N_t}{\mathbf{Y}}_g^{last}$ {Instantaneous CSI-based Users Selection and Digital Precoder Design} 20: while$n\le N_f$do 21:     while $b\le N_{RF,g}$ do 22:         while $k\le K_g$ do 23:             Compute $U\left({\mathbf{F}}_n^{DB}\right)=U\left({\mathbf{F}}_n^{DB}\right)+U\left({\mathbf{f}}_{k^{\prime },n}^{DB}\right),\forall k^{\prime }\in {\mathcal{K}}_g$ 24:             $k^{{}^{\prime }*}=arg\underset{k^{\prime }}{max}\left\{U\left({\mathbf{F}}_n^{DB}\right)\right\}$ 25:             ${\mathsf{\Psi }}_{k^{{}^{\prime }*},n}=1$ 26:             Update ${\mathbf{F}}_n^{DB}$ and $U{\left({\mathbf{F}}_n^{DB}\right)}^{update}$ with user $k^{{}^{\prime }*}$ 27:             if $U{\left({\mathbf{F}}_n^{DB}\right)}^{updated}$≥$U{\left({\mathbf{F}}_n^{DB}\right)}^{last}$ then 28:                 ${\mathcal{K}}_n={\mathcal{K}}_n\cup \left\{k^{\prime *}\right\}$ 29:                 ${\mathcal{K}}_g={\mathcal{K}}_g-\left\{k^{\prime *}\right\}$ 30:                 $U{\left({\mathbf{F}}_n^{DB}\right)}^{last}=U{\left({\mathbf{F}}_n^{DB}\right)}^{updated}$ 31:                 $k++$ 32:             else 33:                 break 34:             end if 35:         end while 36:         $b++$ 37:     end while 38:     $n++$ 39: end while 40: Stack the beamforming matrices ${\mathbf{F}}^{DB}=[{\mathbf{F}}_1^{DB},\dots ,{\mathbf{F}}_{N_f}^{DB}]$ 41: Output 42: ${\mathbf{F}}^{AB}=[{\mathbf{F}}_1^{AB},\dots ,{\mathbf{F}}_G^{AB}]$ 43: ${\mathbf{F}}^{DB}=[{\mathbf{F}}_1^{DB},\dots ,{\mathbf{F}}_{N_f}^{DB}]$ 44: $[{\mathcal{K}}_1,\dots ,{\mathcal{K}}_{N_f}]$

The Algorithm 1 is illustrated in flowchart Figure 5.

5. Machine Learning: K-Means Based Optimal Users Grouping for Analog Beamforming

In this section, we use machine learning technique to group the users. Then, the DFT based fixed switched-beams are used to realize the analog beamforming matrix. The joint users scheduling and hybrid beamforming architecture with ML-based users grouping is shown in Figure 6.

Machine learning algorithms can broadly be divided into two main categories, namely supervised learning and unsupervised learning algorithms. The former class of algorithms learn by training on the input labeled examples, called training dataset, $\{(x^{\left(1\right)},y^{\left(1\right)}),(x^{\left(2\right)},y^{\left(2\right)}),(x^{\left(3\right)},y^{\left(3\right)}),\dots ,(x^{\left(m\right)},y^{\left(m\right)})\}$, where the $i^{th}$ example $(x^{\left(i\right)},y^{\left(i\right)})$ consists of the $i^{th}$ instance of feature vector $x^{\left(i\right)}$ and the corresponding label $y^{\left(i\right)}$. Given a labeled training dataset, these algorithms try to find the decision boundary that separates the positive and negative labeled examples by fitting a hypothesis to the input dataset. Unsupervised machine learning algorithms, on the other hand, are given an unlabeled input dataset. These algorithms are used for extracting information or features from the dataset. These features might be related, but not confined, to the underlying structures or patterns in the input data, relationships in data items, grouping/clustering of data items, etc. Discovered features are meant to provide a deeper insight into the input dataset that can subsequently be exploited for achieving specific goals. Clustering algorithms make an important part of unsupervised learning where the input examples are grouped into two or more separate clusters based on some features. The K-Means (KM) algorithm, is probably the most popular clustering algorithm. It is an iterative algorithm that starts with a set of initial centroids given to it as input. During each iteration, it performs the following two steps.

• Assign Cluster: For every user, the algorithm computes the distance between the user and every centroid. The user is then associated to the cluster with the closest centroid. During this step, a user might change its association from one cluster to another one.
• Recompute centroids: Once all users have been associated to their respective cluster, the new position of centroid for every cluster is then calculated.

Figure 7a depicts how the cluster centroids keep moving across iterations until the system stabilizes for an example network consisting of thirty users being grouped in five clusters. The system becomes stable in only five iterations and the final cluster layout is shown in Figure 7b.

Let us define the following notations to be used later in this section.

$$\begin{array}{cc}\hfill K& =\mathrm{Total}\mathrm{number}\mathrm{of}\mathrm{clusters}\mathrm{being}\mathrm{formed}.\hfill \\ \hfill x^{\left(i\right)}& =\mathrm{Location}\mathrm{coordinates}\mathrm{of}\mathrm{user}{u}^{\left(i\right)}.\mathrm{In}\mathrm{our}\mathrm{case},x^{\left(i\right)}\in \mathrm{I}{\mathrm{R}}^2\hfill \\ \hfill c^{\left(i\right)}& =\mathrm{Cluster}\mathrm{to}\mathrm{which}\mathrm{the}\mathrm{user}{u}^{\left(i\right)}\mathrm{is}\mathrm{currently}\mathrm{associated}.\hfill \\ \hfill {\mu }_k& =\mathrm{Centroid}\mathrm{of}{k}^{th}\mathrm{cluster},{\mu }_k\in \mathrm{I}{\mathrm{R}}^2\hfill \\ \hfill {\mu }_{c^{\left(i\right)}}& =\mathrm{Centroid}\mathrm{of}\mathrm{the}\mathrm{cluster}\mathrm{to}\mathrm{which}\mathrm{the}\mathrm{user}{u}^{\left(i\right)}\mathrm{is}\mathrm{currently}\mathrm{associated}.\hfill \end{array}$$

Now the cost function J can be defined as

$$J(c^{\left(1\right)},c^{\left(2\right)},\dots ,c^{\left(m\right)},{\mu }_1,{\mu }_2,\cdots ,{\mu }_K)={\displaystyle \frac{1}{m}}\sum _{i=1}^m\left|\right|x^{\left(i\right)}-u_{\left(c^i\right)}{\left|\right|}^2$$

(57)

with the following optimization objective function.

$$\begin{array}{c}\hfill \underset{c^{\left(1\right)},,\cdots ,c^{\left(m\right)},{\mu }_1,\cdots ,{\mu }_K}{min}J(c^{\left(1\right)},c^{\left(2\right)},\dots ,c^{\left(m\right)},{\mu }_1,{\mu }_2,\cdots ,{\mu }_K)\end{array}$$

It may be pointed out that Equation (57) allows us to compare multiple clustering layouts based on their cost and select the one with the lowest cost. The above optimization objective function constitutes a non-convex and NP-hard problem because it has many possible local minima and integer optimization variable $c^{\left(i\right)}$. The KM algorithm heuristically optimize this function by alternate minimization method. It iterates between two steps (Assign cluster and Recompute centroids) as described above.

In this section, we use the KM algorithm for optimal clustering of m users competing for resources in a particular cell. The clustering is performed based on their geographic location, thus our input dataset $\{u^{\left(1\right)},u^{\left(2\right)},u^{\left(3\right)},\dots ,u^{\left(m\right)}\}$ has m vectors $u^{\left(i\right)},1\le i\le m$, consisting of location coordinates, of ith user. For the sake of simplicity, we assume these users are deployed in a two dimensional area, i.e., a plane and so $u^{\left(i\right)}=(x_1^{\left(i\right)},x_2^{\left(i\right)})$, i.e., an ordered pair of location coordinates. Our clustering algorithm is summarized in Algorithm 2.

The proposed algorithm takes the location coordinates of m users as input. It also takes two numbers $mi{n}_k$ and $ma{x}_k$ as additional input. The algorithm outputs the best number of clusters, k, such that $mi{n}_k\le k\le ma{x}_k$, and corresponding members of each cluster. It starts with $k=mi{n}_k$ and randomly selects k user locations as the initial centroids (line 6). It assigns the closest centroid to each user (line 8) and then computes new centroids by calculating the center/average location of all nodes in each cluster (line 11). So, in effect, the location of centroids keeps moving in successive iterations. It repeats the above two steps until the change in centroid positions is zero or negligible. We repeat the test $ma{x}_t$ times with a new set of randomly chosen initial centroids every time. During every test, the discovered centroids, corresponding centroid assignment to users, and the cost are saved (lines 14–16) for later comparison. After running the loop for $ma{x}_t$ times, we select and store the best k centroids resulting from the test with the lowest cost while discarding the remaining (lines 19–21). The same is repeated for the next value of k, i.e., $k=k+1$, until $k>ma{x}_k$. At the end we have $cnt=ma{x}_k-ma{x}_k$ vectors ${\mu }_k$, one for each value of k, the corresponding assignment vector ${\mathbf{a}}_k$ and cost $c_k$. Finally, we choose the vector $\mu$ having the lowest cost and corresponding assignment vector $\mathbf{a}$ among $cnt$ stored cases. That is the best number of clusters and corresponding centroids that the algorithm found.

Algorithm 2 K-Means based users grouping algorithm.

1: $cnt=0$   2: for$k=mi{n}_{k}ma{x}_k$do   3:     $cnt=cnt+1$   4:     for $t=1:ma{x}_t$ do   5:         repeat   6:             Randomly choose initial k centroids ${\mu }_1,{\mu }_2,{\mu }_3,\dots ,{\mu }_k$   7:             for $i=1:m$ do   8:                 $a^{\left(i\right)}=j,1\le j\le k$, such that ${\mu }_j$ is the centroid closest to $u^{\left(i\right)}$   9:             end for 10:             for $l=1:k$ do 11:                 ${\mu }_l=$ mean of all users/points $u^{\left(i\right)}$ assigned to $lth$ centroid 12:             end for 13:         until converges 14:         ${\mu }^{\left(t\right)}=({\mu }_1,{\mu }_2,{\mu }_3,\dots ,{\mu }_k)$ 15:         ${\mathbf{a}}^{\left(t\right)}=(a^{\left(1\right)},a^{\left(2\right)},a^{\left(3\right)},\dots ,a^{\left(m\right)})$ 16:         $c^{\left(t\right)}=cost({\mu }_1,{\mu }_2,{\mu }_3,\dots ,{\mu }_k)$ 17:     end for 18:     $idx={arg}_{min}\{c\left(t\right),1\le t\le ma{x}_t\}$ 19:     ${\mathit{\mu }}_{\mathbf{k}}^{\left(k\right)}={\mathit{\mu }}^{\left(idx\right)},1\le idx\le ma{x}_t$ 20:     ${\mathbf{a}}_{\mathbf{k}}^{\left(k\right)}={\mathbf{a}}^{\left(idx\right),}1\le idx\le ma{x}_t$ 21:     $c_k^{\left(k\right)}=c^{\left(idx\right)},1\le idx\le ma{x}_t$ 22: end for 23: $index={arg}_{min}\{c_k^{\left(k\right)},1\le k\le cnt\}$ 24: $\mathit{\mu }={\mathit{\mu }}_{\mathbf{k}}^{\left(index\right)},1\le index\le cnt$ 25: $\mathbf{a}={\mathbf{a}}_{\mathbf{k}}^{\left(index\right)},1\le index\le cnt$ 26: $n=index$

After the groups formation, BS sends this information to all users, where users use this information to form reduced average statistical CSI. For example, a user in a group of 5, needs to send the average statistical CSI only after $1/5$ of regular feedback interval time.

6. Simulation Results

Consider the downlink of a multiuser massive MIMO single cell with three 120 degree sectors. We neglect inter-sector interference and focus on a single 120 degree sector served by a ULA of $N_t=64$ isotropic antennas at BS. The users grouping forms virtual sectors inside 120 physical sector.

In simulation, the results are obtained by averaging over 100 drops. In each drop we randomly generate spatial correlation matrices ${\mathbf{R}}_g$. For each realization of spatial correlation matrix ${\mathbf{R}}_g$, we simulate 1000 realizations of instantaneous channel $\mathbf{H}$.

The joint spatio–radio scheduling and hybrid precoder scheme first forms the users groups and then selects the beams that maximizes the sum-rate through downlink training process. Secondly, it calculates the ZF based digital precoder using low dimensional effective channel feedback from the users.

Figure 8 shows the CDF of the non-zero eigenvalues of channel covariance matrix. Notice that approximately $50\%$ of the non-zero eigenvalues are close to zero. The sum-rate increases as the number of groups increases at the cost of increased feedback overhead as shown in Figure 9. Using machine learning technique in Section 5 we can get optimal number of groups from channel covariance feedback. This results in increased sum-rate with substantial reduced feedback. The optimal $G=3$ gives $27.6\%$ increase in sum-rate compared to when $G=1$ and $62.5\%$ decrease in feedback overhead compared to $G=8$. The comparison of performance of ML-based users grouping with previous work cannot be provided because there is no previous work that uses ML-based technique to reduce the CSI feedback overhead in massive MIMO systems. Many papers use users grouping in massive MIMO hybrid beamforming [3,5,39,40], but they do not utilize ML-based users grouping. Therefore, we have compared our proposed solution with two benchmarks of full-CSI ($G=K$) and coarse-CSI ($G=1$). Figure 10 shows sum-rate with number of users at $10dB$ SNR. For a fixed number of groups $G=3$, the increase in number of users, increases number of users per group. Due to the fixed number of groups, the feedback overhead remains the constant. Sum-rate is increasing with users because we assumed $N_{RF}=N_s=K$. If we fix the number of RF chains to some hardware limit, then the sum-rate will saturate at specific number of users. It can be seen in Figure 10, that increasing number of users per group decreases the slop of the sum-rate for limited CSI schemes. This decrease is due the increase in intra-group interference.

Sum-rate also depends on the number of RF chains but this dependence is not linear as shown in Figure 11. This figure shows sum-rate variation with number of RF chains $N_{RF}$ when $N_s=8$, $K=8$, $N_t=64$, and $SNR=$ 10 dB. Sum rate increases with number of RF chains because it yields better conditioned effective channel matrix. It can be seen that the spectral efficiency does not increase monotonically with $N_{RF}$ and saturates at $N_{RF}=N_t$ where hybrid precoding is turned to the pure digital precoding. The increase in spectral efficiency with the number of RF chains comes at the cost of higher dimensional effective channel feedback overhead and power consumption in RF chains.

The spectral efficiency of the proposed scheme also varies with number of transmit antennas as shown in Figure 12. In the figure, $N_{RF}=N_s=K=8$, $SNR=10$ dB, and BS has 16, 64, 128 or 256 ULA antennas. The performance gain increases with the increase in number of transmit antennas because large antennas array increases the resolution of the transmit beams (also depicts in figure 4) and, hence, decreases the potential of inter-beams interference.

In general, the spectral efficiency is a function of SNR and for the $SNR=10$ dB, our ML-based users grouping and hybrid beamforming scheme gives $27.6\%$ increased sum-rate at the cost of $33.3\%$ extra feedback overhead as compared to the coarse-CSI case (G = 1). Our proposed scheme incurs $62.5\%$ reduced feedback at the cost of $25.2\%$ reduction in sum-rate as compared to the full-CSI case (G = K).

7. Conclusions

This paper studied the limited feedback two-stage hybrid beamformimg for decomposing the precoding matrix at the base-station. The huge channel state information feedback of massive MIMO has been reduced by the channel covariance-based RF precoding and beam selection. The well-known regularized block diagonalization can mitigate the inter-group interference, but requires substantial feedback. We used K-mean algorithm based unsupervised machine learning technique for users grouping and channel covariance-based eigenmodes/discrete Fourier transforms to reduce the feedback overhead and designed a simplified analog precoder. The digital precoder is designed with joint optimization of intra-group user utility function. It has been shown that more than $50\%$ feedback overhead is reduced by the eigenmodes-based analog precoder design. The spatio–radio resources scheduling and limited feedback-based hybrid precoding increases the sum-rate by $27.6\%$ compared to the sum-rate of one-group case at the cost of $33.3\%$ extra feedback overhead, and reduces the feedback overhead by $62.5\%$ at the cost of $25.2\%$ reduction in sum-rate, compared to the full CSI feedback.