Robust Hybrid Beam-Forming for Non-Orthogonal Multiple Access in Massive MIMO Downlink

Abstract

This paper presents hybrid precoding for a non-orthogonal multiple access (NOMA) transmission scheme in a millimeter wave (mmWave) massive MIMO (mMIMO) downlink. In hybrid precoding, the analog precoder is obtained by the orthogonalization of the users’ channel vectors to minimize inter-beam interference. The digital precoder consists of a zero-forcing precoder to minimize inter-user interference. In order to break the barrier of one user per beam, we utilize the NOMA within the beam for power domain multiplexing among users. Simulation results show the proposed scheme’s efficacy compared to the state-of-the-art schemes and provide 1.48 times better sum-rate performance at 10 dB received SNR.

1. Introduction

The huge demand for various services and the rapid growth of mobile Internet and Internet of Things (IoT) applications have led to research on fifth-generation (5G) communication systems [1]. There are two prominent frequency bands in 5G communications: the millimeter wave (mmWave) frequency band and the sub-6 GHz frequency band. By operating at 30–300 GHz, mmWave communication may provide orders-of-magnitude higher bandwidths than sub-6 GHz band communication. Hence, the fifth-generation (5G) wireless communication requirement for high throughput motivates the wireless industry to use mmWave communication for its vast bandwidth advantage [2]. Aside from the wide bandwidth, the smaller wavelengths in mmWave allow more antennas to be packed into the same physical space, allowing vast multiple-inputs multiple-outputs to be supported more effectively in mMIMO [3,4]. This also introduces spatial multiplexing and diversity gains. In fact, it has been shown that mmWave-mMIMO can increase system capacity by orders of magnitude [5].

However, the implementation of mmWave mMIMO in practice still faces challenges [6]—to fully exploit the gains provided by mMIMO, each antenna needs a dedicated radio frequency (RF) chain. Due to space limitations, it is challenging to implement mMIMO systems. However, the mmWave frequency range made it possible, as at such high frequencies, the antenna size is drastically reduced. Many antennas also result in an equal number of RF chains, which are too expensive and power-hungry. The user equipment (UE) and the base station (BS) use phased array beamforming technology to solve this problem. Beam-forming is a technique that can control an array of antennas to transmit radio signals in a specific direction. Antenna arrays using beamforming not only transmit power or signals in all directions but also determine the direction of interest and send and receive more substantial signal beams in that specific direction [7].

The beamforming architecture is mainly divided into digital beamforming, analog beamforming, and hybrid beamforming [8]. In digital beamforming, amplitude/phase shift keying is applied to the digital signal (digital to analog converter (DAC)) at the transmitting end. The opposite process is done after analog to digital converter (ADC) and digital downlink converter (DDC) operations; digital beamforming includes RF translators, A/D conversion, DDC, complex weight multiplication, and a summing function. In comparison to digital beamforming, analog and hybrid beamforming architectures can minimize the number of RF chains by combining analog or mixed analog-digital signal processing. In particular, analog beamforming is usually implemented using phase shifter networks and is one of the simplest ways to apply MIMO in mmWave systems. However, its performance is affected by the quantitative phase shift and lack of amplitude matching. Hybrid beamforming (HB) combines analog and digital beamforming; it uses each benefit and may acquire excellent stability among system complexity and overall performance by accurately deciding on various RF chains. Hybrid beamforming is classified into partially connected HB architectures and fully connected architectures, as shown in Figure 1. The partially connected HB uses a separate subset of a phase shifter and antenna array with each RF chain. Because of the structure, this is also called sub-array hybrid beamforming. In fully connected HB architecture, the RF beamformer of each RF circuit uses all antennas; this is where the RF signals from various RF circuits are combined before being sent through the antennas. Due to the architecture, this type is also called full array HB [9]. In a fully interconnected architecture, additional RF aggregation components can cause failures and performance losses, and it is one of the main problems of mmWave radio. On the other hand, the partial connection architecture of the RF chain allows fewer antennas to be accessed, which leads to severe shortcomings such as large beam widths, fewer directions, and significant interference from other chains. They try to reduce the interference to the partially connected HB type. The partial connection architecture provides a simpler circuit and lower loss than the fully connected architecture [6,9]. The fully-connected structure provides a more significant array gain.

The maximum number of users that mmWave mMIMO can handle falls as the number of RF chains decreases. Non-orthogonal multiple access (NOMA) has recently been added to mmWave mMIMO to break this essential restriction and expand the number of concurrently supported users [10,11]. NOMA, unlike traditional orthogonal multiple access (OMA), uses superposition coding and successive interference cancellation (SIC) to allow several users to access the same time-frequency resource [12,13]. Furthermore, because of the very directed nature of mmWave transmission, the usage of NOMA in mmWave is advantageous since the users’ channels can be strongly linked [14].

In heterogeneous networks, NOMA has proven to be a promising technique to increase the performance of multi-tier networks. A two-tier network consists of Macro-base stations (MBS) and small-base stations (SBS). Parihar et al. [15] examined the use of cooperative NOMA to achieve SWIPT in heterogeneous networks. Further, [16] provides unique time slotting solutions that attempt to boost the throughput of weak users while ensuring user fairness, which is especially important as the number of users in a NOMA group rises. The coexistence of multi radio access technology (RAT) with ultra-high frequency SBS and mmWave SBS layers in heterogeneous networks is studied in [17]. At the SBS level, NOMA is employed to handle a high number of users. To establish the order of the users for NOMA, the SBS must obtain the user’s CSI. Getting CSI with SBS is challenging since the number of users significantly increased. The suggested approach employs double-restricted feedback from the user. The first is for user orders, while the second is for selecting the appropriate RAT. A three-tier NOMA is employed in a heterogeneous cellular network that includes an MBS layer, a Femto base station (FBS) tier, and a device-to-device (D2D) tier, with NOMA being used in the FBS and D2D levels [18]. Reference [19] presents a smarter HetNet with an MBS and FBS layer. The FBS layer is equipped with NOMA and carrier sensing support for its transmissions. Carrier detection helps reduce interference by preventing base stations within a specific transmitter range from broadcasting and increasing performance.

The sparse massive MIMO channel has few multipath components, and the channel matrix is ill-conditioned [20]. This study aims to investigate a low complexity hybrid precoding for a non-orthogonal multiple access (NOMA) transmission scheme in a real-World ill-conditioned mmWave massive MIMO downlink. The remainder of this paper is structured in the following manner. Section 2 presents related work. The system, signal and channel model are covered in Section 3. In Section 4, the proposed energy-efficient massive MIMO-NOMA is discussed. The proposed hybrid beamforming NOMA algorithm is explained in Section 5. The simulation results are presented in Section 6. Finally, Section 6 includes a conclusion.

Notations: Bold upper/lower case letters denote vectors and matrices, respectively. The notations $X^{-1}$, $X^T$, $X^H$, and $tr\left(X\right)$ denote the inverse, transpose, Hermitian (conjugate transpose), and trace of a matrix $X$. $diag\mathit{p}$ is the diagonal entries of a matrix $P$, |$ℬ$| is the cardinality of set$ℬ$, ${ℂ}^{\mathit{M}\times \mathit{N}}$ denotes the set of all $M\times N$ matrices with complex entries, and $\mathcal{C}\mathcal{N}\left(\mathsf{\mu },{\mathsf{\sigma }}^2\right)$ is the circularly symmetric complex Gaussian distribution with mean $\mathsf{\mu }$ and variance ${\mathsf{\sigma }}^2$. All the acronyms are defined in

Table 1. List of acronyms.

Acronyms	Description	Acronyms	Description
5G	Fifth-generation	MM	Maximum magnitude
ADC	Analog-to-digital converter	mMIMO	Massive MIMO
AoD	Angle of departure	MMSE	Minimum mean squared error
AWGN	Additive white Gaussian noise	mmWave	Millimeter wave
BS	Base station	MPCs	Multi path components
CSI	Channel state information	MU-MIMO	Multi-User MIMO
DAC	Digital-to-analog converter	NLOS	Non line-of-sight
DDC	Digital downlink converter	NOMA	Non-orthogonal multiple access
DFT	Discrete Fourier transforms	OFDM	Orthogonal frequency division multiplexing
DoA	Direction of arrival	OMA	Orthogonal multiple access
DoD	Direction of departure	RF	Radio frequency
DoF	Degree of freedom	SE	Spectral efficiency
DPC	Dirty paper coding	SIC	Successive interference cancellation
EVD	Eigenvalue decomposition	SINR	Signal-to-interference-plus-noise ratio
FDD	Frequency division duplex	SVD	Singular value decomposition
GS	Gram-Schmidt	SVM	Support vector machine
HB	Hybrid beamforming	TDD	Time division duplex
HH	House-holder	TDMA	Time division multiple access
IoT	Internet of Things	UE	User equipment
LOS	Line-of-sight	ULA	Uniform linear array
MIMO	Multiple-input multiple-output	ZF	Zero-forcing

.

2. Related Work

Reference [21] proposes and investigates a multi-beam NOMA strategy for the mmWave mMIMO system and also studies its resource allocation systems. Each radio frequency chain utilizes a beam splitting mechanism to create several analog beams to serve diverse NOMA consumers. The proposed approach may enable NOMA transmission for users with arbitrary departure angles, unlike the traditional mmWave-NOMA technology, which can only support multiple users in a single beam. This gives NOMA in mmWave communications more flexibility, allowing it to take advantage of the possible multi-user diversity. The authors devised a suboptimal two-stage beam-user allocation strategy in order to optimize the system’s sum rate. To optimize the conditional system sum rate in the first stage, a user grouping and antenna allocation technique based on coalition formation game theory were used with known analog beamforming. The second stage employs the ZF digital precoder to solve a nonconvex optimization problem for the power allocation with the objective of optimizing the system sum-rate while taking quality-of-service constraints into account. A beam splitting system provides many analog beams for NOMA transmission to satisfy distinct clients; instead of the present mmWave orthogonal multiple access (mmWave OMA) systems, the suggested solution might service numerous users on each RF chain. Unlike the conventional mmWave-NOMA method, which can accommodate multiple NOMA users inside the same beam, the suggested system may execute NOMA transmission for users with any AoD distribution. This increases the flexibility of NOMA, allowing it to better exploit the potential for multi-user diversity. The simulation results demonstrated that, in each stage, the intended resource allocation might provide near-optimal results. This multi-beam NOMA scheme is designed under the assumption of sparsely populated users such that there is a very low probability that two or more users come under the same beam. The beam gain is also decreased when one RF chain is associated with two or more spatially apart users. In addition, the beam width is increased with the reduced number of antennas per RF chain.

A user categorization technique based on channel gains and correlations is suggested in [22]. Joint hybrid beamforming and power allocation problems are devised to optimize the possible sum rate. To tackle this issue, first they form users’ groups using the K-means method and then they find a power allocation method for arbitrary fixed hybrid beamforming. They use a boundary-compressed particle swarm optimization technique to solve the analog beamforming issue and use the zero-forcing form to construct the digital beamforming to reduce inter-group interference. To begin addressing this nonconvex problem with high-dimensional variables, the authors obtain the power allocation solution for arbitrary fixed hybrid beamforming, which is divided into intra-group and inter-group power allocation. Then, using the approximation zero-forcing technique, they create the digital beamforming to decrease inter-group interference given arbitrarily set analog beamforming. To implement NOMA with HBF, it utilizes the K-means algorithm and the normalized channel correlation for the users’ grouping. Users with low channel correlation are assigned to the same group, while users with high channel correlation are assigned to separate groups, significantly lowering interference between groups of users. After that, the power allocation and HBF design problem is formulated to optimize the users’ average sum rate, with each user having a minimum rate constraint. This paper does not discuss inter-beam interference and suffers from a higher computational complexity.

The author of [23] considers that each user uses analog-only beamforming, whereas the BS uses a combination of analog and digital beamforming. Several users can share the same beam via the NOMA technique to achieve a higher cumulative capacity. However, because distinct localized users’ analog beamformers are used, the BS’s digital beamformers are not fully aligned with the users’ baseband effective channels. Several users may be allocated similar or even identical beamformers. Meanwhile, a user pairing and power allocation mechanism is developed to minimize interference from other beams and increase the aggregate capacity. After users’ pairing, it uses a matched filter based analog beamformer, which is simple but it may suffer from inter-group interference, whereas our proposed scheme provides orthogonal beams which eliminate the inter-group interference.

Wei et al. [24] eliminate the antenna allocation part from [21] by increasing the beamwidth to accommodate more users in a NOMA group. More crucially, by extending the beamwidth, the suggested beamwidth control may increase the number of serviced NOMA groups, thus maximizing the energy efficiency benefit provided by NOMA. Two beamwidth control approaches are given to this purpose, one based on conventional beamforming and the other on Dolph–Chebyshev beamforming. The authors first analyze the primary lobe power losses caused by the two beamwidth management approaches and suggest an efficient analog beamformer design to reduce the power loss. This paper investigates a non-orthogonal multiple access (NOMA) technique based on beamwidth management. The proposed technology permits various customers in a single NOMA set to share the same radio frequency chain and analog beam for superposition transmission. In this article, a beamwidth control approach is investigated to overcome the physical limits of the narrow analog beam by extending the diameter of the analog beamwidth to smooth the construction of NOMA clusters. The authors then discuss the suggested bandwidth control’s significant lobe power loss and derive the asymptotically optimum analog beamformer from enhancing the system’s sum in the antenna bulk mode. The recommended beamwidth control-based NOMA strategy is compared against a baseline scheme that uses time division multiple access to determine overall system improvements (TDMA). The authors focus on a simple method for adjusting the beamwidth of an analog beam by turning off some antennas to achieve this goal. To maximize the system sum-rate in a wide variety of antennas regimes, they designed the asymptotically optimal analog beamformer after studying the significant lobe power loss related to beamwidth management. However, widening the beam width causes inter-group interference. Though the proposed scheme has less computational complexity than [21], it suffers from low user diversity gain.

Table 2. Related work comparison.

Paper	Analog Beamforming	Digital Beamforming	NOMA Type	Complexity	Inter-Group Interference Mitigation	Intra-Group Interference Mitigation
Wei et al. [21]	AoD-based	ZF	LOS AoD and chahhel gain based	high	ZF digital precoder	Beam splitting usinf antenna allocation
Zhu et al. [22]	Particle swarm	ZF	Channel correlation-based K-mean	high	Approx. ZF	SIC
Almasi et al. [23]	Matched filter	ZF	Channel correlation-based	low	ZF of strong user	SIC
Wei et al. [24]	Dolph-Chebyshev	MMSE	Game theory	low	Dolph-Chebyshev analog precoder	SIC

shows the summary of the related work.

Contribution

In this paper, we develop robust hybrid beamforming for NOMA-based millimeter-wave massive MIMO systems. We focus on the joint advantage of the low complexity analog precoding and system capacity enhancement using NOMA-mMIMO system. We use a real-World mmWave mMIMO channel with a strong LOS component. For this type of channel, it is extremely difficult to get the channel inversion matrix. This type of channel matrix is called an ill-conditioned channel matrix. We use the Householder orthogonalization method to get the analog precoding matrix as it has minimum orthogonality error with the ill-conditioned matrix. We use the stronger user channel vector as the group channel vector, which eliminates the inter-group interference in NOMA groups. The contributions of this paper are summarized as follows:

• Robust hybrid beamforming is designed by using the House-holder based channel orthogonalization method. The above-mentioned related works use either SVD-, EVD-based analog beams, or the fixed beam structure (DFT). Though DFT-based analog beamforming is simple, and many researchers used DFT-based fixed analog beamforming, but this further requires a computation-intensive beam-user pairing process.
• We use the strongest user’s channel vector as the group channel. In the case of the ill-conditioned channel matrix, the proposed scheme provides low orthogonality error and renders almost orthogonal analog beams. Due to the orthogonal beams, inter-group interference is eliminated.
• Simulation is performed in the more realistic wireless environment, which is controlled by the channel matrix condition number. It can be seen that the proposed scheme requires less transmit power to guarantee the users’ target spectral efficiency as compared to the SVD, EVD, and GS-based NOMA-mMIMO schemes.

3. System, Signal, and Channel Model

This section presents the system, signal, and channel model for the proposed massive MIMO hybrid beamforming and NOMA downlink system.

3.1. System and Signal Model

In the conventional multiuser MIMO systems, the combined received signal

y = [y₁, y₂, …, y_K] ^T can be expressed as:

y = H^H FP_S + v,

(1)

where s = [s₁, s₂, …, s_K ]^T is the K × 1 transmitted signal for all K users with normalized power E(ss^H)= I_K, P = diag{p} is the total transmitted power for K users and

p = [√(p₁),√(p₂),…,√(p_K)],

(2)

satisfies ${\displaystyle \sum }_{K=1}^K{p}_k \le P$ (the maximum transmitted power at the BS), F = [f₁, f₂, …, f_K ] is the N_t × K beamforming matrix with ‖f_k‖₂ = 1 for k = 1,2,…,K, and v is the AWGN noise with zero mean and variance ${\sigma }^2$, that is, $𝒞$$𝒩$(0, σ²I_K). H = [h₁, h₂, …, h_K] of size N_t × K is the channel matrix, where h_k of size N_t × 1 denotes the MIMO channel vector between BS and kth user. Finally, the superscript H is a Hermitian matrix, which is a complex square matrix that is equal to its own conjugate transpose of the channel matrix (H). Particularly, in this paper, we consider the widely used Saleh–Valenzuela channel model for the mmWave communication, which is explained in the following subsection.

The number of needed RF chains at the BS in conventional MIMO systems is equal to the number of transmit antennas, that is, N_RF = N_t, which is typically huge for a MIMO systems, for example, N_RF = N_t = 256. As a result, due to high hardware costs and energy consumption caused by RF chains, the conventional implementation of massive MIMO at mmWave frequencies is prohibitive. Each RF chain, for example, consumes roughly 250 mW, and a mmWave massive MIMO system with 256 antennas requires 64 W. To overcome this problem, the concept of beam space MIMO was recently used, in which a lens antenna array is used to minimize the required number of RF chains in mmWave large MIMO systems to reduce the power consumption.

The lens antenna array’s role is to realize the spatial domain analog beamforming using N_t × N_t discrete Fourier transform matrix U, as follows:

$$\mathrm{U}={[\mathrm{a}\left({\overline{\mathsf{\theta }}}_1\right), \mathrm{a}\left({\overline{\mathsf{\theta }}}_2\right), \dots \dots , \mathrm{a}\left({\overline{\mathsf{\theta }}}_{{\mathrm{N}}_{\mathrm{t}}}\right)]}^{\mathrm{H}},$$

(3)

where ${\overline{\mathsf{\theta }}}_{{\mathrm{N}}_{\mathrm{t}}}=\frac{1}{{\mathrm{N}}_{\mathrm{t}}}\left(n-\frac{{\mathrm{N}}_{\mathrm{t}}+1}{2}\right)$ for n = 1, 2, …, N_t are the fixed spatial directions. Then, the received signal vector $\overline{y}$ can be represented as:

$$\overline{y}=\overline{H}FPs+v,$$

(4)

where the channel matrix $\overline{H}$ is defined as:

$$\overline{H}=UH=\left[U{h}_1, U{h}_2, \dots \dots ,U{h}_k \right]=\left[{\overline{h}}_1,{\overline{h}}_2,\dots ,{\overline{h}}_K\right],$$

(5)

where ${\overline{h}}_K=U{h}_k$is the beam domain channel vector of the kth user.

Then,

$$\overline{y}=H^H{U}^{H}FPs+v.$$

(6)

There are N_t beams with spatial directions ${\overline{\mathsf{\theta }}}_1, {\overline{\mathsf{\theta }}}_2, \dots , {\overline{\mathsf{\theta }}}_{{\mathrm{N}}_t}$ as each row of the beam domain channel matrix $\overline{H}$ in (5), corresponds to one beam. At mmWave frequencies, there is a limited scattering and most of the NLoS component die after a few reflections, therefore, the number of NLoS components which actually reach the receiver is much smaller than the number of beams, that is, $L\ll N_t$. Since, there are only a few dominant elements in each beam domain channel vector ${\overline{h}}_K$, therefore, the beam domain channel matrix $\overline{H}$ is a sparse matrix. The beam selection algorithm exploits this sparse structure to design reduced-dimension beam domain MIMO systems without significant performance loss. Because of the sparse nature of the beam domain channel matrix, we can use some beam selection algorithm to select K out of N_t beams to serve K users at the same time. Then, we can write the received signal in (6) as:

$$\overline{y}={\overline{\mathit{H}}}_{\mathit{B}}{}^H{F}_{B}Ps+v,$$

(7)

where ${\overline{\mathit{H}}}_{\mathit{B}} \in {ℂ}^{|ℬ|\times \mathit{K}}$ is the reduced-dimension beam domain channel matrix after the beam’s selection, and $ℬ$ is the set of selected beams. $F_B$ of the size |$ℬ$| × K is the reduced-dimension beamforming matrix. By exploiting the sparse nature of the mmWave channel, we can have a significantly reduced number of required RF chains, and we need only N_RF = |$ℬ$| RF chains. Existing beam domain MIMO systems, on the other hand, can only handle one user at a time. As a result, the maximum number of users in a given frequency resource cannot exceed the number of RF chains, that is, K ≤ N_RF, which is the ultimate bound of beam domain MIMO systems. This is because the DoF required by users must be less than or equal to the DoF of RF chains; otherwise, the linear operation would fail to separate signals for different users. To solve this constraint, we propose a new joint analog precoding and beam selection approach that combines NOMA and massive MIMO in the following section.

3.2. Channel Model

The extended Saleh–Valenzuela mathematical model is used to simulate the mmWave Channel because, mathematically, it precisely captures the propagation pattern found in mmWave channels [25]. Let there be N_C scattering clusters around the transmitter and receiver, such that every cluster produces L_p multipath components [26]. At mmWave frequencies, the electromagnetic propagation waves exhibit high vapor absorption and low penetration strength; therefore, there are limited multipath components in each cluster u ∈ {1, …, N_C} and a dominant line-of-sight (LOS) wave propagation. When scattering clusters are randomly situated, measurements have shown that the Saleh–Valenzuela model is a good fit for describing the random behavior of the MPC’s amplitude, phase and arrival delay in the mmWave channel [27]. Empirical measurements show that the mmWave channel fits the extended Saleh–Valenzuela model [28]. The N_t x 1 channel vector of user 𝑘 can be written as in [29]:

$${\mathit{h}}_{\mathit{k}}=\sqrt{\frac{N_t}{N_{C}Lp+1} } {{\displaystyle \sum }}_{u=1}^{N_C}{{\displaystyle \sum }}_{l=0}^{Lp} {\alpha }_k^{u_{,}l} a\left({\varphi }_k^{U,l}\right)e^{-j2\pi un/N_t} ,$$

(8)

where j = √−1, ${\alpha }_k^{u_{,}0}$ and ${\alpha }_k^{u_{,}l}$, l = 1, …, L^p represent the complex gain of the LOS, l = 0 path, and non-line-of-sight (NLOS), l = 1, …, L^p paths are independent and identically distributed with $𝒞$$𝒩$ (0,1) in cluster 𝑢, respectively. a is the transmit array response vector. The variable ${\varphi }_k^{U,l}$ is the azimuth angles (the aiming angle in the array) of arrival in cluster u [30] for user 𝑘 with uniform distribution, ${\varphi }_k^{U,l}$~$𝒰$ (−π/2, π/2). The transmit steering vector can be expressed as:

$$a\left({\varphi }_{\mathrm{k}}^{\mathrm{U},\mathrm{l}}\right)=\frac{1}{\sqrt{N_t}} {\left[\exp (-j2\pi \frac{d}{\lambda } \sin {\varphi }_{\mathrm{k}}^{\mathrm{U},\mathrm{l}} i)\right]}_{\mathrm{i}ϵI\left(N_t\right)}^T,$$

(9)

where I(N_t) = {i−(N_t−1)/2, i = 0, 1, …, N_t−1} is the set of indices [31], λ is the wavelength of the propagation waves and d is the spacing between the antenna elements at the base-station. At the end, we get the general channel matrix H of the multiuser MIMO system as, H = [h₁, ……, h_k] ∈ 𝕔^NtxK.

4. Proposed Energy-Efficient Massive MIMO-NOMA

In the proposed MIMO-NOMA method, unlike previous mMIMO systems, more than one user per beam can be serviced concurrently, as shown in Figure 2. In simple massive MIMO beamforming, each beam is produced by an RF chain, and beam selection methods such as maximum magnitude (MM) [21] and signal-to-interference-plus-noise-ratio (SINR) maximization [7] can be used to choose one beam for each user. It is worth mentioning that there is a high probability that different users choose the same beam, a phenomenon known as “conflicting users” in this type of problem.

In a typical mmWave mMIMO system with N_t = 128 antennas and K = 16 users and the uniformly distributed spatial directions, the chance of users picking the same beam is 62.4% [32].

Unlike existing large MIMO systems, which use user scheduling to select only one user from a group of competing users, the proposed MIMO-NOMA system can serve conflicting users simultaneously using the same RF chain. Despite the fact that the beam selection algorithm can select the beams up to N_RF, the number of concurrently serviced users, K, can be greater than N_RF, that is, K ≥ N_RF. Let S_n be the set of users served by the nth beam for n = 1, 2, …, N_RF and one user can be served by only one beam, that is, S_i ⋂ S_j = Φ for i ≠j and $\sum _{n=1}^{N_{RF}}\left|S_n\right|=K$. After beam selection for the mth user in the nth beam, the N_RF×1 beam domain channel vector is represented by h_m,n, and N_RF×1 f_n shows the uniform precoding vector for users in the nth beam. We can assume that $\Vert {\mathit{h}}_{1,n}^H {\mathit{f}}_n{\Vert }_2\ge \Vert {\mathit{h}}_{2,n}^H {\mathit{f}}_n{\Vert }_2 \ge \dots \ge \Vert {\mathit{h}}_{\left|S_n\right|,n}^H {\mathit{f}}_n{\Vert }_2 for n=1, 2, \dots , N_{RF}$. The received signal at the kth user in the nth beam can be expressed as:

$$y_{k,n}={\mathit{h}}_{k,n}^H {\displaystyle \sum }_{J=1 }^{N_{RF}}{\displaystyle \sum }_{I=1}^{\left|S_j\right|}{\mathit{f}}_j\sqrt{p_{i,j}{s}_{i,j} }+v_{k,n}$$

(10)

$$\begin{gathered} \underset{\mathrm{desired} \mathrm{signal}}{\underbrace{={\mathit{h}}_{k,n}^H{\mathit{f}}_n\sqrt{p_{k,n} }{s}_{k,n}}} \underset{\mathrm{intra}-\mathrm{beam} \mathrm{interferences}}{\underbrace{+{\mathit{h}}_{k,n}^H {\mathit{f}}_{n }{{\displaystyle \sum }}_{i=1}^{m-1 }\sqrt{p_{i,n} }{s}_{i,n}+{\mathit{h}}_{k,n}^H {\mathit{f}}_{n }{{\displaystyle \sum }}_{i=k+1}^{\left|S_n\right| }\sqrt{p_{i,n} }{s}_{i,n} }} \\ +\underset{inter-beam interference}{\underbrace{{\mathit{h}}_{k,n }^H{{\displaystyle \sum }}_{j \ne n }{{\displaystyle \sum }}_{i=1}^{\left|S_j\right|}{\mathit{f}}_j\sqrt{p_{i,n}}{s}_{i,j}}}+\underset{noise}{\underbrace{v_{k,n}}}, \end{gathered}$$

(11)

where p_k,n is the transmitted power and s_k,n is the transmitted signal for the kth user in the nth beam, and v_k,n is the AWGN noise with zero mean and variance ${\sigma }^2$. In order to apply SIC decoding at the receiver, more power is allocated to the user with a low channel-to-noise ratio (CNR), and less power is allocated to the user with a high CNR. The user with a high CNR first decodes the signal of a low CNR user and then removes that signal from the composite received signal to get its signal. The low CNR user directly decodes its signal while considering the other user’s signal as simple noise. In the case of two users in a NOMA group with sorted CNR as $CN{R}_1>CN{R}_2$, then, the power allocation would be $p_1<p_2$. Note that in existing beam domain MIMO systems, only one user can be supported in each selected beam, that is, |S_n| = 1(n = 1, 2, …, N_RF), whereas in the proposed MIMO-NOMA system, |S_n| can be larger than one for n = 1, 2, …, N_RF.

In Equation (11), the precoding vectors ${\left\{{\mathit{f}}_{\mathrm{n}}\right\}}_{\mathrm{n}=1}^{{\mathrm{N}}_{\mathrm{RF}}}$ should be cautiously designed to minimize the inter-beam interferences. Due to the NOMA, intra-beam interferences are induced by the power domain multiplexing of users within a beam [33,34]. It can be suppressed by conducting SIC in the order of increasing equivalent channel gains, that is, by performing SIC, the kth user in the nth beam eliminates the interferences from the ith user ($\forall$i > k) in the nth beam. In massive MIMO systems, NOMA allows more than one user to be serviced at the same time within each beam, and this allows us to schedule the number of users greater than the number of selected analog beams, that is, K ≥ N_RF. In the suggested MIMO-NOMA system, however, users are subjected to interferences from other beams in addition to intra-beam interferences. As a result, in practice, a simple combination of NOMA with MIMO cannot ensure consistent performance and precoding; power allocation should be devised to minimize interferences by maximizing the sum spectral efficiency.

Digital beamforming of the system model is given by:

$$\begin{gathered} H_{eff}=F_{RF} h_k \\ {F}_{BB}=H_{eff}{\left(H_{eff}{}^H{H}_{eff}+I\left(K\right)\right)}^{-1}. \end{gathered}$$

(12)

Calculating hybrid beamforming of the model based on the Householder method is given by:

$$F_{HBF}=F_{RF} F_{BB}.$$

(13)

The received SINR at user k in beam n is given by:

$$\begin{array}{l}SIN{R}_{k,n}\\ =\frac{p_{k,n}{\left|h_{k,n}^H{f}_n\right|}^2}{{\left|h_{k,n}^H f_{n }\right|}^2{{\displaystyle \sum }}_{i=1}^{m-1 }{p}_{i,n}+{\left|h_{k,n}^H f_{n }\right|}^2{{\displaystyle \sum }}_{i=k+1}^{\left|S_n\right| }{p}_{i,n}+{\left|h_{k,n }^H\right|}^2{{\displaystyle \sum }}_{j \ne n }{{\displaystyle \sum }}_{i=1}^{\left|S_j\right|}{f}_j{p}_{i,n}+{\sigma }_v^2 }\end{array}$$

(14)

and the total power transmit power is given by:

$$P={{\displaystyle \sum }}_{n=1}^N{{\displaystyle \sum }}_{k=1}^{K_n}\frac{2^{\left(R_{target}\right)}-1}{SIN{R}_{k,n}},$$

(15)

where $K_n$ is the number of users in beam n, and R_target is the target information rate per user.

The system sum rate is given by:

$$R_{sum}={{\displaystyle \sum }}_{n=1}^N{{\displaystyle \sum }}_{k=1}^{K_n}{\log }_2\left(1+P_{k,n}SIN{R}_{k,n}\right).$$

(16)

The Proposed Hybrid Beamforming Noma Algorithm

The steps of the proposed algorithm are described in detail as follows:

(1)

The algorithm initializes first the parameter values of the users and channel.

(2)

Compute the channel matrix of the mmWave massive MIMO systems.

(3)

A zero-forcing precoder is used to calculate the mmWave digital beamforming to minimize interference between users.

(4)

Calculate the analog beamforming using different mathematical methods to minimize the inter-beam interference by orthogonalization of the user’s channel vectors.

(5)

Add both analog and digital beamforming to get hybrid beamforming.

(6)

This paper adds NOMA into beam domain mmWave massive MIMO systems to further enhance spectrum efficiency and connectivity density. Unlike the existing beam domain MIMO system, two or more users can be simultaneously accommodated within each selected beam in Algorithm 1.

Algorithm 1. Low Complexity Hybrid Beamforming with NOMA

Input: Channel State Information, Total power

Output: carrierToNoiseRatio, PowerAllocated of analog and digital beamforming

1 Initializes NOMA parameter values: Nt, K, NRF, subChannels, bandwidth, initial Power

2 Calling mmWave channel model function: 2.1. Calling Initialize function. 2.2. Generate hk→(Nt × 1)-dimension for each user k 2.3. Implementing NOMA (i.e., two pathloss value to get users with different channel gain in each beam):    For 1 to length of user group (ρk)     hk = sqrt(Nt/(Ls × ρk (i))) × (ak × (z.T)); 2.4. Get desired condition number CondNumb = 10 × sqrt(2); →Desired condition number For 1to length of user group (ρk) Singular value decomposition of the channel matrix (h(Nt × K)) H = U × S × VT

3 Initialization: assign zero to variable SINR

4 Calling House Holder function: Input: Channel model or channel matrix Output: Analog beamforming and digital beamforming 4.1. Initialize parameters →assign zero to analog and digital matrix 4.2. Find the reflector: 4.3. Apply Householder reflection to each column of A (channel matrix) and Q (identity matrix of Nt): i.e., Nt is the number of antennas and K number of the user. for j from 1 to K A = A(k to Nj)−v × (2 × (vT × A(k to Nj))); end for j from 1 to Nt Q = Q(k to N j)−v × (2 × (vT × Q(k to Nj))); end 4.4. Computing Analog Beamforming (FRF): Q = QT; R = upperTriangular(A); Qtrancated = Q(1 to K); for i from 1 to K for j from 1 to N FRF = (1/sqrt(Nt)) × (Qtrancated(j,i)/sqrt(Qtrancated(j,i) × Qtrancated(j,i)T); end end 4.5.  Computing Digital Beamforming (FBB): FBB using Equation (12)

5 Calculate hybrid beamforming-based householder (FHBF): FHBF using Equation (13)

6 Initialization→assign zero to Sum Rate (Rsum).

7 Calculate carrier to noise ratio (SINR): For k to users (K) Assign interference to zero→interference = 0 Assign noise to some value→noise = ${10}_{-6}$ SINRk = (abs(FHBF(k)T × hk(k)))2 (SINR); end

8 Compute the transmit power for target rate (P (dBm)): P using Equation (15)

9 Compute Sum Rate (Rsum): Rsum using Equation (16)

Computational complexity: For a $N_t\times K$ channel matrix, the SVD operation has a complexity of $\mathcal{O}\left(N_t{K}^2\right)$. The SVD is required to include the desired condition number in the real-world channel realization. The Householder orthogonalization based analog beamformer and user grouping complexity is $\mathcal{O}\left(2N_t{K}^2-\frac{2K^3}{3}\right)$. The ZF-based digital precoder requires matrix multiplication and matrix inversion operations with the complexity $\mathcal{O}\left(N_t{K}^2+2K^3\right)$ and $\mathcal{O}\left(K^3\right)$, respectively. Hence, the complexity of the above algorithm is of the order of $\mathcal{O}\left(2N_t{K}^2-\frac{2K^3}{3}\right)$.

5. Simulation Results

The suggested low complexity hybrid beamformer based on the Householder function (HHHBF) is assessed in this section. The proposed scheme is compared with the Gram–Schmidt-based hybrid beamforming (GSHBF), eigenvalue decomposition-based hybrid beamforming (EVDHBF), and singular value decomposition-based hybrid beamforming (SVDHBF) in terms of transmit power performance against the target SE. The digital-only beamformer for both NOMA and OMA are also included in the graphs as a benchmark.

We examine a single cell with Nt = 128 uniform linear array (ULA) antenna elements and N_RF = 8 RF chains on the base station and K = 16 single-antenna users. The inter-element distance in the ULA is λ/2, where is the transmission wavelength. The ill-conditioned MIMO channel matrix is described using the logarithmic condition numbers 0 dB and 11.5 dB. In the simulations, the number of multipath components L = 5 is used. The findings are achieved by averaging over 1000 channel realizations in simulation.

Figure 3 depicts the transmit power required to achieve the desired target SE in a well-conditioned massive MIMO channel. The required transmit power increases with the increasing target SE. The dotted-line graphs are for various OMA schemes, and the solid lines are for the NOMA schemes. It can be seen that all NOMA have low power consumption as compared to the respective OMA schemes for a target SE. It can be noted that in the well-conditioned channel, GS-based NOMA and HH-based NOMA give an identical performance; because both GS and HH are channel orthogonalization based analog beamforming methods. The HHHBF and GSHBF give similar performance to that of the digital beamformer. The HHHBF performs better in the ill-conditioned channel condition due to its low orthogonality errors than the GSHBF method, as shown in Figure 4.

Figure 5 shows the transmit power required with the increasing number of users when the target SE is 5 bits/s/Hz. It can be seen that HH-based HBF NOMA provides 15 dBm gain over the corresponding HH-based HBF OMA scheme. We take the number of users from eight to sixteen in order to show the effect of the OMA (when K = 8) to two users per group NOMA (when K = 16). As the number of users increases, the required transmit power to maintain the target SE is not increased linearly because of the users’ diversity gain. Again, both GS-based and HH-based schemes are showing an identical performance in the well-conditioned massive MIMO channel, which is similar to the optimal digital beamforming.

The real-world ill-conditioned massive MIMO channel is used to evaluate the performance of the proposed HH-based HBF NOMA scheme in Figure 5. Our proposed solution performed well in the realistic channel model, and the performance gap between the proposed scheme and the ideal digital beamforming is close to zero. This graph shows the strength of our proposed scheme compared to GS-based HBF NOMA, EVD-based HBF NOMA, and SVD-based HBF NOMA. Additionally, when we compare the HH-based HBF NOMA with HH-based HBF OMA, we can see that their performance gap increases with the increasing target SE because increasing SE requires more power and/or bandwidth. OMA just divides the bandwidth and power among the competing user; however, NOMA uses power domain multiplexing to achieve the demand of the competing users.

In Figure 6, the required transmit power is plotted against the number of users in the ill-conditioned massive MIMO channel with the target SE of 5 bits/s/Hz. The NOMA digital beamforming has the lowest transmit power for all users. The proposed HH-based HBF NOMA scheme has a better performance and low transmit power than the GSHBF NOMA scheme due to low orthogonality error with sparse and ill-conditioned channel matrix. HHHBF also provides a minimum performance gap with the ideal digital beamforming.

Finally, we plot the average sum-rate with the received SNR in the ill-conditioned channel in Figure 7. Sum-rate is the logarithmic function of the received SNR. The SVD- and EVD-based HBF NOMA schemes produce a low sum-rate performance because common scatterers in the mmWave channel result in a similar angle-of-arrival at the BS and cause severe interuser interference. However, the channel orthogonalization based HBF (GS-based and HH-based) eliminates the interference in the analog beam domain. In the case of the real-world ill-condition channel, the HHHBF NOMA outperforms the GSHBF NOMA because of the low orthogonality error in the analog domain. It can be seen that the proposed HH-based HBF NOMA produces a 1.48 times better performance than the GS-based HBF NOMA at 10 dB received SNR.

6. Conclusions

In this paper, we present an HH-based hybrid beamforming design for a mmWave massive MIMO NOMA system. The analog precoder is obtained by the orthogonalization of the channel matrix using householder reflectors. Due to the orthogonal analog beamforming, inter-group interference is eliminated in the NOMA schemes. In addition, the householder orthogonalization provides a low orthogonality error in the case of ill-conditioned channel matrices. The proposed HH-based BF NOMA outperforms the state-of-the-art schemes and gives a 1.48 times better sum-rate performance at 10 dB compared to the GS-based HBF NOMA and gives a sum-rate performance close to the ideal digital beamforming NOMA. In potential future work, we will add the beam-user pairing to form the optimize NOMA groups for QoS constrained energy-efficient performance. In NOMA systems, the strong users have to keep the CSI of weak users in order to apply the SIC successfully, which incurs extra processing and a CSI load on the strong user. As a possible future work, deep reinforcement learning can be used to alleviate the strong user and increase its energy efficiency.

7. Patents

This section is not mandatory but may be added if there are patents resulting from the work reported in this manuscript.