Transceiver Optimization for mmWave Line-of-Sight MIMO Systems Using Hybrid Arrays

Abstract

The performance of millimeter wave (mmWave) line-of-sight multiple input multiple output (LOS MIMO) systems using hybrid arrays of planar subarrays was studied. We characterized the achievable maximum spatial multiplexing gain for such LOS MIMO systems by the measures of spectral efficiency and effective degree of freedom (EDoF). By proposing a joint plane-wave and spherical-wave-based general 3D channel model, we derived the optimal design parameters in the analog domain, i.e., the optimal subarray separation products, and analyzed their sensitivity on the system performance. We also gave analytical eigenvalue expressions of the equivalent LOS MIMO channel matrix, which are applicable to the case of a non-optimal design, as well as the upper and lower bounds of the EDoF for system performance evaluation. A piecewise uniform quantization codebook was further designed for quantizing phase shifter values in practical applications. The numerical and simulation results show that planar subarrays are superior to traditional arrays in terms of spectral efficiency and EDoF in Ricean fading channels because they are more robust to the change in the communication distance and the deviation from the optimal design. The use of hybrid arrays of planar subarrays effectively removes the limitation of mmWave LOS MIMO systems using traditional arrays, through which, the conventional Rayleigh distance criterion has to be satisfied to achieve the optimal performance.

1. Introduction

With the ever increasing demand for data traffic, network throughput and wireless devices, traditional microwave frequency bands have been unable to satisfy the requirements of a fast-growing system capacity and spectral efficiency [1]. Therefore, the millimeter wave (mmWave) with a higher frequency band has recently attracted more attention [2,3,4]. The mmWave communications generally operate at 30 GHz–300 GHz with a wavelength of 1 mm–10 mm, which leads to a relatively small physical size of the antennas. This enables the large-scale multiple input multiple output (MIMO) with hundreds of antennas that can meet the high throughput and coverage needs of future networks to be integrated in a limited space for practical applications [5], and also to compensate for the high path loss of mmWave channels for long-distance communications [6,7].

Unlike the traditional MIMO channels in dense scattering environments, mmWave channels are usually characterized as sparse scattering, and thus go against conventional spatial multiplexing [8,9]. Since the pure line-of-sight (LOS) component generally dominates the mmWave channels, it is of great significance to study LOS MIMO spatial multiplexing by employing well-designed array deployment [10]. Hybrid antenna arrays have proven to be a critical technology of cost-effectively achieving high-capacity and long-range communications [11,12,13]. They will not only enable LOS MIMO without the need for adjusting the spacing of subarrays, but also significantly increase the transmission power and decrease the propagation loss by beamforming to reach longer communication distances [14]. The related works are not yet to be widely developed in LOS MIMO systems using hybrid arrays of analog planar subarrays.

In LOS channels, the reflective paths do not exist or their power can be ignored, which results in a higher probability of an insufficient rank in the MIMO channel matrix. However, a well-designed antenna array can still be used to obtain a high-rank channel matrix and higher spatial multiplexing gain in a pure LOS environment [15]. References [16,17] enhance the spatial multiplexing gain of LOS MIMO systems by adjusting the array diameter in a uniform circular array. The technique based on the optimization of antenna placement in uniform linear digital arrays with respect to mutual information was investigated in [18,19], and then extended to uniform planar arrays in [20]. The optimal antenna separation product was derived for a general 3D geometrical LOS MIMO channel model. It was found that the optimal performance can be obtained only when the system parameters satisfy the Rayleigh distance criterion. In general, this refers to the condition that the communication distance R should be equal to or less than the so-called Rayleigh distance, i.e., $R\le V{d}_t{d}_{r}cos{\theta }_{t}cos{\theta }_r/\lambda$, where $d_t{d}_r$ is the antenna separation product, $\lambda$ the carrier wavelength, V the maximum number of antennas between the transceiver and ${\theta }_t$ (${\theta }_r$) the angles of the local spherical coordinate system at the transmitter (receiver).

Although LOS MIMO has been considered to be able to achieve spatial multiplexing, the conventional full digital array LOS MIMO technology is infeasible for use in aerial platforms as the antenna spacing would have to be adjusted according to the communication distance, which is ever-changing. The inflexible adjustment is likely to cause a serious performance loss of spatial multiplexing when the Rayleigh distance criterion cannot be satisfied. To reduce the optimal antenna spacing for achieving full spatial multiplexing, the investigation in [21] added a dielectric medium to the signal transmission path for potentially improving channel conditioning with additional phase shifting. Although the limitation of the Rayleigh distance criterion can be relieved to a certain extent, each change in the transmission distance has to be accompanied by the replacement of dielectric mediums with different shapes and types, which is inflexible and even infeasible in practical applications. The work in [22] introduced an asymmetric linear subarray structure instead of the traditional digital arrays only at the transmitter, which can provide both array gain and spatial multiplexing gain and achieve a trade-off between them. In [23], the design of multiple planar arrays at the transceiver was investigated in LOS MIMO systems, and an algorithm for optimizing the antenna position was proposed to reduce the computational complexity. However, only the spherical wave model was used for the LOS channel component, and the phase shifters matrix was not considered for the beamforming design.

Considering the power consumption and system complexity, radio frequency (RF) analog beamformer/combiners are typically implemented using phase shifters. The phase shifters are controlled digitally, and a finite number of values is usually selected subject to the quantization bits. Various quantization codebook designs or phase shifter structures have been investigated in mmWave MIMO systems [24,25,26]. The study in [24] considered two kinds of RF beamforming codebooks. One is the general quantized beamforming codebook, which is usually designed for rich channels with a uniform quantization on the space of beamforming vectors. The other is the beamsteering codebooks, where the beamforming vectors as spatially matched filters can be parameterized by a simple angle. Its simulation results showed the effect of the number of quantization bits on the spectral efficiency in limited scattering mmWave channels. A two-phase-shifter structure for each coefficient of the RF analog precoder was proposed and analyzed in [25], through which, any precoding coefficients can be represented with a very small quantization error when the number of quantization bits is sufficiently large. To reduce the large quantization error introduced by separated quantization, a joint quantization method that uses a combined codebook of the two phase shifters was proposed in [26]. Although the two-phase-shifter structure can control the beamforming weight flexibly to achieve full multiplexing gain, the doubled number of phase shifters will increase the complexity of hardware implementation. A codebook-based beamforming training method was used for RF precoding weights of subarrays in [22], whereas the impact of codebooks and quantization bits on the system performance was not given.

In this paper, we studied the performance of mmWave LOS MIMO systems employing hybrid arrays of planar subarrays in terms of the spectral efficiency and effective degree of freedom (EDoF), where the EDoF represents the achievable maximum system spatial multiplexing gain. We propose a joint plane-wave and spherical-wave-based general 3D channel model for the LOS MIMO channel, which allows for a non-parallel orientation of hybrid arrays to be employed at the transceiver. The performance in terms of the spectral efficiency and EDoF using hybrid arrays of planar subarrays was analyzed and compared with that of the traditional digital arrays for different deviation factors and Ricean fading channels. The numerical and simulation results show that using the proposed planar subarray structure can still achieve an almost optimal performance even though the parameter design deviates from the Rayleigh distance criterion in a relatively wide range. The main contributions can be summarized as follows.

• Based on the proposed 3D channel model, the optimal subarray separation products in the vertical and horizontal directions were derived for maximizing the spectral efficiency and EDoF, respectively, and an analysis of the sensitivity to the non-optimal design was performed by means of deviation factors.
• The theoretical expressions for the eigenvalues of an equivalent LOS MIMO channel matrix were derived as a function of deviation factors, in both cases of optimal and non-optimal designs. The upper and lower bounds of the EDoF are also given.
• The piecewise uniform quantization codebooks for the phase angles in the beamformer and combiner were designed for LOS MIMO systems, which enables the results to be applied in practical systems with quantized RF phase shifters. Analytical expressions for spectral efficiency using the designed codebooks were also derived. The numerical results demonstrate that the designed codebooks outperform the beamsteering codebook in [24] using the same number of quantization bits in terms of spectral efficiency.

The remainder of this paper is organized as follows. Section 2 introduces the received signal models using hybrid arrays with planar subarrays and performance measures, followed by a joint plane-wave and spherical-wave-based 3D channel model. Section 3 derives the eigenvalue expressions of the equivalent channel matrix considering deviation factors caused by a non-optimal design, and gives the upper and lower bounds of EDoF. Section 4 designs uniform quantization codebooks for phase shifter values employed in practical applications. In Section 5, numerical and simulation results are given to verify our analysis and demonstrate the effects of design parameters on the system performance, before concluding the paper in Section 6.

The following notations are used throughout this paper. ${\mathbf{A}}^H$ denotes the conjugate transpose of matrix $\mathbf{A}$; ${∥\mathbf{A}∥}_F$ is the Frobenius norm of matrix $\mathbf{A}$; ${\mathbf{1}}_U$ is the $U\times U$ all-ones matrix; ${\mathbf{I}}_U$ is the $U\times U$ identity matrix; $\mathrm{diag}\left(\mathbf{a}\right)$ denotes a diagonal matrix whose diagonal elements are formed by vector a; ${∥\mathbf{a}∥}_2$ is the 2-norm of vector $\mathbf{a}$, and ${\mathbf{a}}^T$ is its transpose; $\mathrm{E}\left\{·\right\}$ is used to denote expectation. Further, the notations $\det \left\{·\right\}$, $\mathrm{tr}\left\{·\right\}$ and $\mathrm{rank}\left\{·\right\}$ represent the determinant, trace and rank of $\left\{·\right\}$, respectively.

2. System Model

2.1. Signal Model

Consider a point-to-point LOS MIMO communication system where the hybrid arrays with planar subarrays are used in the transceiver, respectively. As shown in Figure 1, each subarray is connected by an RF chain with multiple phase shifters, which are used to implement analog beamforming. Let the number of subarrays at the transmitter (receiver) be $N=N_{ty}\times N_{tx}$ ($M=M_{ry}\times M_{rx}$), where $N_{ty}$ and $N_{tx}$ ($M_{ry}$ and $M_{rx}$) represent the numbers of vertical and horizontal subarrays, respectively. We assume that each subarray at the transmitter (receiver) contains $P\times P$ ($Q\times Q$) antenna elements. Denote the transmitted signal vector as $\mathbf{s}\in {\mathbb{C}}^{N\times 1}$ and the normalized complex channel matrix as $\mathbf{H}\in {\mathbb{C}}^{M{Q}^2\times N{P}^2}$. Let $\mathbf{F}\in {\mathbb{C}}^{N{P}^2\times N}$ and $\mathbf{W}\in {\mathbb{C}}^{M{Q}^2\times M}$ denote the phase shifter matrices at the transceiver, respectively. Assuming slowly varying and frequency-flat fading channels (although we only study the narrowband channels in this paper, the proposed approach can be extended to wideband channels by dividing the wideband into multiple narrowbands, e.g., in OFDM systems. The optimal design and analysis in this paper can be applied to each narrowband subcarrier accordingly), the received signal vector $\mathbf{y}\in {\mathbb{C}}^{M\times 1}$ after analog beamforming combining for each subarray can be modeled as

$$\begin{array}{c}\hfill \mathbf{y}=\sqrt{\rho }{\mathbf{W}}^H\mathbf{H}\mathbf{F}\mathbf{s}+{\mathbf{W}}^H\mathbf{n}=\sqrt{\rho }\tilde{\mathbf{H}}\mathbf{s}+{\mathbf{W}}^H\mathbf{n},\end{array}$$

(1)

where $\rho$ is the power attenuation coefficient of the transmitted signals over a subchannel between a pair of transmit and receive subarrays, and assumed to be a constant for all subchannels [27,28]. $\mathbf{n}\in {\mathbb{C}}^{M{Q}^2\times 1}$ is the additive white Gaussian noise (AWGN) vector with zero mean value and noise power ${\sigma }_n^2$. $\tilde{\mathbf{H}}\in {\mathbb{C}}^{M\times N}$ denotes the equivalent baseband channel between the transceiver after the LOS MIMO propagation and the analog beamforming.

Assuming that the channel state information is perfectly known at the receiver and equal power transmission is used, the spectral efficiency of the system can be expressed as [29]

$$\begin{array}{c}\hfill SE={log}_{2}det\left({\mathbf{I}}_U+{\displaystyle \frac{\overline{\gamma }}{N}}\mathbf{V}\right)=\sum _{k=1}^U{log}_2\left(1+{\displaystyle \frac{\overline{\gamma }}{N}}{\mu }_k\right)\mathrm{bits}/\mathrm{s}/\mathrm{Hz},\end{array}$$

(2)

where $\overline{\gamma }$ represents the average SNR after analog beamforming combining at the receiver and $U=min\{M,N\}$. ${\mu }_k$ is the kth eigenvalue of matrix $\mathbf{V}$, and $\mathbf{V}$ is expressed as

$$\mathbf{V}=\left\{\begin{array}{cc}\tilde{\mathbf{H}}{\tilde{\mathbf{H}}}^H,\hfill & M<N\\ {\tilde{\mathbf{H}}}^H\tilde{\mathbf{H}},\hfill & M\ge N.\end{array}\right.$$

(3)

The MIMO system can achieve the optimal performance only when its channel is equivalent to multiple independent single input single output (SISO) subchannels. Therefore, we characterize the achievable spatial multiplexing gain of the system by maximizing the EDoF as [23,30]

$$\begin{array}{c}\hfill EDoF={\displaystyle \frac{d\left(SE\left(2^{\delta }\overline{\gamma }\right)\right)}{d\delta }}{|}_{\delta =0}=\sum _{k=1}^U{\displaystyle \frac{\overline{\gamma }{\mu }_k}{N+\overline{\gamma }{\mu }_k}}.\end{array}$$

(4)

From (2) and (4), we can see that the spectral efficiency and EDoF are not only determined by the eigenvalue ${\mu }_k$ of matrix $\mathbf{V}$, but also by the number of subarrays at the transmitter and the average SNR at the receiver. When the product of the eigenvalue ${\mu }_k$ and the average SNR is far greater than N, a 3 dB increase in SNR leads to an approximate increase in U bits/s/Hz in spectral efficiency [30]. The work in [19] revealed that the optimal EDoF and system spectral efficiency can be achieved when any two columns or rows of the channel matrix satisfy orthogonality, i.e., all eigenvalues of the channel matrix are equal. This orthogonality criterion will be used in Section 3.

2.2. Channel Model

A Ricean channel matrix can be modeled as the sum of an LOS component and an NLOS component in [31,32]. Assume that the normalized channel matrix $\mathbf{H}$ is Ricean and can be expressed as

$$\begin{array}{c}\hfill \mathbf{H}=\sqrt{\frac{K}{1+K}}{\mathbf{H}}_{LOS}+\sqrt{\frac{1}{1+K}}{\mathbf{H}}_{NLOS},\end{array}$$

(5)

where the Ricean factor K represents the ratio between the power of the two components. In this paper, we focused on the pure LOS channel in an MIMO system, which indicates that $K=+\infty$. ${\mathbf{H}}_{LOS}$ is hereinafter referred to as $\mathbf{H}$ unless stated otherwise. As shown in [33], when the antenna array is appropriately configured, the pure LOS MIMO channel matrix is a high-rank matrix with a number of nonzero eigenvalues, which leads to a high spectral efficiency.

As illustrated in Figure 2, we assume that the separations of adjacent subarrays along the vertical and horizontal directions at the transmitter (receiver) are $S_{tx}$ and $S_{ty}$ ($S_{rx}$ and $S_{ry}$), respectively, and that the spacings of adjacent antenna elements in each subarray along the vertical and horizontal directions are d for simplicity. The transmit arrays are placed on the $xy$ plane, $\beta$ is the angle between the $xy$ plane and the plane of the receive arrays and R represents the horizontal communication distance between the bottom of the transceiver arrays. As in [34,35], due to $d\ll (S_{tx},S_{ty},S_{rx},S_{ry})\ll R$, we used the plane-wave model for the received signals within a subarray, but the spherical-wave model for the received signals between different subarrays, which is reflected in the distance between a pair of antenna elements at the transceiver.

Therefore, an element of the channel matrix $\mathbf{H}$, $h_{r,t}$ ($r=0,1,\dots ,M{Q}^2-1$, $t=0,1,\dots ,N{P}^2-1$), in the planar subarray structure can be represented as

$$\begin{array}{c}\hfill h_{r,t}=e^{j\frac{2\pi }{\lambda }{l}_{r,t}},\end{array}$$

(6)

where $\lambda$ is the carrier wavelength, $l_{r,t}$ represents the distance between the tth transmitting antenna element and the rth receiving antenna element and

$$\begin{array}{c}\hfill r=(m_y{M}_{rx}+m_x)Q^2+y_{mr}Q+x_{mr},\end{array}$$

$$\begin{array}{c}\hfill t=(n_y{N}_{tx}+n_x)P^2+y_{nt}P+x_{nt},\end{array}$$

(7)

where $m_x\left(m_y\right)=0,1,\dots ,M_{rx}-1(M_{ry}-1)$, $n_x\left(n_y\right)=0,1,\dots ,N_{tx}-1(N_{ty}-1)$, $x_{mr}\left(y_{mr}\right)=0,1,\dots ,Q-1$ and $x_{nt}\left(y_{nt}\right)=0,1,\dots ,P-1$. Let $m=m_y{M}_{rx}+m_x$ and $n=n_y{N}_{tx}+n_x$. $l_{r,t}$ can be rewritten as [36]

$$\begin{array}{cc}\hfill l_{r,t}=& l_{m,n}+d·x_{nt}sin{\theta }_{nt}cos{\phi }_{nt}+d·y_{nt}sin{\theta }_{nt}sin{\phi }_{nt}\hfill \\ \hfill & +d·x_{mr}sin{\theta }_{mr}cos{\phi }_{mr}+d·y_{mr}sin{\theta }_{mr}sin{\phi }_{mr},\hfill \end{array}$$

(8)

where $l_{m,n}$ represents the distance between a reference element in the nth transmit subarray and that in the mth receive subarray. In this paper, we took the antenna element located at the origin of the coordinate axis in the first subarray as the reference element. ${\theta }_{nt}$ and ${\phi }_{nt}$ (${\theta }_{mr}$ and ${\phi }_{mr}$) are the signal elevation and azimuth angles away from the nth transmit subarray (to the mth receive subarray), respectively. $l_{m,n}$ is further calculated by

$$\begin{array}{cc}\hfill l_{m,n}=& {\left[{(m_x{S}_{rx}-n_x{S}_{tx})}^2+{(R+m_y{S}_{ry}sin\beta )}^2+{(m_y{S}_{ry}cos\beta -n_y{S}_{ty})}^2\right]}^{\frac{1}{2}}\hfill \\ \hfill =& (R+m_y{S}_{ry}sin\beta ){\left[1+\frac{{(m_x{S}_{rx}-n_x{S}_{tx})}^2+{(m_y{S}_{ry}cos\beta -n_y{S}_{ty})}^2}{{(R+m_y{S}_{ry}sin\beta )}^2}\right]}^{\frac{1}{2}}\hfill \\ \hfill \stackrel{\left(a\right)}{\approx }& R+m_y{S}_{ry}sin\beta +\frac{{(m_x{S}_{rx}-n_x{S}_{tx})}^2+{(m_y{S}_{ry}cos\beta -n_y{S}_{ty})}^2}{2R},\hfill \end{array}$$

(9)

where $\left(a\right)$ holds due to $R\gg (S_{tx},S_{ty},S_{rx},S_{ry})$ by using the McLaughlin approximation formula; that is, when $x\to 0$, ${(1+x)}^{1/a}=1+{\displaystyle \frac{1}{a}}x$.

As a result, the channel matrix $\mathbf{H}$ can be expressed as

$$\begin{array}{c}\hfill \mathbf{H}=\left[\begin{array}{ccc}{\mathbf{H}}_{0,0}& \dots & {\mathbf{H}}_{0,(N-1)}\\ ⋮& \ddots & ⋮\\ {\mathbf{H}}_{(M-1),0}& \dots & {\mathbf{H}}_{(M-1),(N-1)}\end{array}\right],\end{array}$$

(10)

where the subchannel matrix

$$\begin{array}{c}\hfill {\mathbf{H}}_{m,n}=h_{m,n}\left[\begin{array}{ccc}{\widehat{h}}_{0,0}& \dots & {\widehat{h}}_{0,(P^2-1)}\\ ⋮& \ddots & ⋮\\ {\widehat{h}}_{(Q^2-1),0}& \dots & {\widehat{h}}_{(Q^2-1),(P^2-1)}\end{array}\right]\end{array}$$

and

$$\begin{array}{cc}\hfill {\widehat{h}}_{q_m,p_n}=& exp(j{d}^{\prime }[q_m^{x}sin{\theta }_{mr}cos{\phi }_{mr}+q_m^{y}sin{\theta }_{mr}sin{\phi }_{mr}\hfill \\ \hfill & +p_n^{x}sin{\theta }_{nt}cos{\phi }_{nt}+p_n^{y}sin{\theta }_{nt}sin{\phi }_{nt}\left]\right),\hfill \end{array}$$

where $q_m=q_m^{y}Q+q_m^x$, $p_n=p_n^{y}P+p_n^x$, $q_m^x\left(q_m^y\right)=0,1,\dots Q-1$, $p_n^x\left(p_n^y\right)=0,1,\dots P-1$ and $d^{\prime }=\frac{2\pi }{\lambda }d$.

As the equivalent channel matrix $\tilde{\mathbf{H}}$ includes the analog beamforming matrix $\mathbf{F}$ and combining matrix $\mathbf{W}$, we need to consider their influence on the optimal design in the analog domain of LOS MIMO systems with planar subarrays. $\mathbf{F}$ and $\mathbf{W}$ can be expressed as

$$\begin{array}{c}\hfill \mathbf{F}=\mathrm{diag}\{{\mathbf{f}}_0,{\mathbf{f}}_1,\dots ,{\mathbf{f}}_{N_{tx}-1},{\mathbf{f}}_{N_{tx}},\dots ,{\mathbf{f}}_{N-1}\}\end{array}$$

(11)

and

$$\begin{array}{c}\hfill \mathbf{W}=\mathrm{diag}\{{\mathbf{w}}_0,{\mathbf{w}}_1,\dots ,{\mathbf{w}}_{M_{rx}-1},{\mathbf{w}}_{M_{rx}},\dots ,{\mathbf{w}}_{M-1}\},\end{array}$$

(12)

respectively. In this paper we used a single beam for each subarray, and therefore the normalized weight vectors ${\mathbf{f}}_n$ and ${\mathbf{w}}_m$ of the nth transmit subarray and the mth receive subarray are written as

$$\begin{array}{cc}\hfill {\mathbf{f}}_n=& \frac{1}{P}[1,\dots ,exp(-j{d}^{\prime }(p_n^{x}sin{\alpha }_{nt}cos{\varphi }_{nt}+p_n^{y}sin{\alpha }_{nt}sin{\varphi }_{nt})),\hfill \\ \hfill & \dots ,exp(-j(P-1)d^{\prime }(sin{\alpha }_{nt}cos{\varphi }_{nt}+sin{\alpha }_{nt}sin{\varphi }_{nt})){]}^T\hfill \end{array}$$

(13)

and

$$\begin{array}{cc}\hfill {\mathbf{w}}_m=& \frac{1}{Q}[1,\dots ,exp\left(j{d}^{\prime }(q_m^{x}sin{\alpha }_{mr}cos{\varphi }_{mr}+q_m^{y}sin{\alpha }_{mr}sin{\varphi }_{mr})\right),\hfill \\ \hfill & \dots ,exp\left(j(Q-1)d^{\prime }(sin{\alpha }_{mr}cos{\varphi }_{mr}+sin{\alpha }_{mr}sin{\varphi }_{mr})\right){]}^T,\hfill \end{array}$$

(14)

which allow the main beam of the nth transmit subarray (the mth receive subarray) to be directed towards the direction represented by the angles (${\alpha }_{nt},{\varphi }_{nt}$) ((${\alpha }_{mr},{\varphi }_{mr}$)).

3. Optimal Design of Planar Subarrays

3.1. Analysis of Eigenvalues

As stated in [19], for a conventional LOS MIMO system with linear digital arrays, the optimal product of the inter-antenna distances at the transceiver can be acquired when different columns or rows of the channel matrix satisfy orthogonality, which is used to maximize the spectral efficiency and EDoF. Inspired by this, we investigated the effect of the planar subarray separations at the transceiver on the system spatial multiplexing performance.

We first considered the situation where M is larger than N. Although the rank of the equivalent channel matrix $\tilde{\mathbf{H}}$ is U in general, only the power allocated to EDoF out of U can be devoted to the spectral efficiency. The correlation among the components of $\tilde{\mathbf{H}}$ becomes increasingly high with the reduced adjacent subarray separations at the transceiver, which can cause a loss of system spectral efficiency and EDoF. In order to reduce the correlation between RF chains, we aimed to design the beamforming matrices such that different columns of $\tilde{\mathbf{H}}$ are orthogonal, i.e., ${\tilde{\mathbf{H}}}_{n_1}^H{\tilde{\mathbf{H}}}_{n_2}=0$ ($n_1\ne n_2$).

From (10), (13) and (14), the orthogonality requires that

$$\begin{array}{cc}\hfill & {\tilde{\mathbf{H}}}_{n_1}^H{\tilde{\mathbf{H}}}_{n_2}\hfill \\ \hfill =& \sum _{m=0}^{M-1}{\mathbf{f}}_{n_1}^H{\mathbf{H}}_{m,n_1}^H{\mathbf{w}}_m{\mathbf{w}}_m^H{\mathbf{H}}_{m,n_2}{\mathbf{f}}_{n_2}\hfill \\ \hfill =& \sum _{m=0}^{M-1}{\displaystyle \frac{1}{P^2{Q}^2}}{h}_{m,n_1}^H{h}_{m,n_2}\sum _{q_m=0}^{Q^2-1}exp[-j{d}^{\prime }(A_m{q}_m^x+B_m{q}_m^y)]\hfill \\ \hfill & ·\sum _{p_{n_1}=0}^{P^2-1}exp[-j{d}^{\prime }(C_{n_1}{p}_{n_1}^x+D_{n_1}{p}_{n_1}^y)]\sum _{q_m=0}^{Q^2-1}exp\left[j{d}^{\prime }(A_m{q}_m^x+B_m{q}_m^y)\right]\hfill \\ \hfill & ·\sum _{p_{n_2}=0}^{P^2-1}exp\left[j{d}^{\prime }(C_{n_2}{p}_{n_2}^x+D_{n_2}{p}_{n_2}^y)\right]\hfill \\ \hfill =& 0,\hfill \end{array}$$

(15)

where

$$\begin{array}{c}\hfill A_m=sin{\theta }_{mr}cos{\phi }_{mr}-sin{\alpha }_{mr}cos{\varphi }_{mr},\end{array}$$

$$\begin{array}{c}\hfill B_m=sin{\theta }_{mr}sin{\phi }_{mr}-sin{\alpha }_{mr}sin{\varphi }_{mr},\end{array}$$

$$\begin{array}{c}\hfill C_{n_k}=sin{\theta }_{n_{k}t}cos{\phi }_{n_{k}t}-sin{\alpha }_{n_{k}t}cos{\varphi }_{n_{k}t}\end{array}$$

and

$$\begin{array}{c}\hfill D_{n_k}=sin{\theta }_{n_{k}t}sin{\phi }_{n_{k}t}-sin{\alpha }_{n_{k}t}sin{\varphi }_{n_{k}t},(k=1,2).\end{array}$$

Since ${\displaystyle \sum _{q_m=0}^{Q^2-1}}exp[-j{d}^{\prime }(A_m{q}_m^x+B_m{q}_m^y)]$ and the last three summations in (15) are not guaranteed to be zero, we let

$$\begin{array}{c}\hfill {\tilde{\mathbf{H}}}_{n_1}^H{\tilde{\mathbf{H}}}_{n_2}=\sum _{m=0}^{M-1}{h}_{m,n_1}^H{h}_{m,n_2}=\sum _{m=0}^{M-1}exp\left(j\frac{2\pi }{\lambda }(l_{m,n_2}-l_{m,n_1})\right)=0.\end{array}$$

(16)

Substituting (9) into (16), we can obtain

$$\begin{array}{cc}\hfill & \sum _{m=0}^{M-1}{h}_{m,n_1}^H{h}_{m,n_2}\hfill \\ \hfill =& \sum _{m_y=0}^{M_{ry}-1}\sum _{m_x=0}^{M_{rx}-1}exp\left(j\frac{2\pi }{\lambda }\frac{\left[m_x(n_{1_x}-n_{2_x})S_{tx}{S}_{rx}\right]}{R}\right)exp\left(j{\displaystyle \frac{2\pi }{\lambda }}\frac{[m_y(n_{1_y}-n_{2_y})S_{ty}{S}_{ry}cos\beta ]}{R}\right)\hfill \\ \hfill =& \frac{sin\left({\displaystyle \frac{M_{ry}\pi }{\lambda R}}(n_{1_y}-n_{2_y})S_{ty}{S}_{ry}cos\beta \right)}{sin\left({\displaystyle \frac{\pi }{\lambda R}}(n_{1_y}-n_{2_y})S_{ty}{S}_{ry}cos\beta \right)}\frac{sin\left({\displaystyle \frac{M_{rx}\pi }{\lambda R}}(n_{1_x}-n_{2_x})S_{tx}{S}_{rx}\right)}{sin\left({\displaystyle \frac{\pi }{\lambda R}}(n_{1_x}-n_{2_x})S_{tx}{S}_{rx}\right)}\stackrel{\left(b\right)}{=}0.\hfill \end{array}$$

(17)

There are infinite solutions for $\left(b\right)$ in (17), but we only chose the smallest subarray separation products in the vertical and horizontal directions as they are of most interest in practical systems. Therefore, the optimal subarray separation products are derived by

$$\begin{array}{c}\hfill S_{ry}{S}_{ty}={\displaystyle \frac{\lambda R}{M_{ry}cos\beta }},S_{rx}{S}_{tx}={\displaystyle \frac{\lambda R}{M_{rx}}},\end{array}$$

(18)

which enable the maximal spatial multiplexing gain of the system with planar subarrays. A similar result can be obtained by substituting $N_{ty}$ and $N_{tx}$ for $M_{ry}$ and $M_{rx}$ in (18) when M is smaller than N. In general, the optimal subarray separation products for planar subarrays can be given by

$$\begin{array}{c}\hfill S_{ry}{S}_{ty}={\displaystyle \frac{\lambda R}{V_{1}cos\beta }},S_{rx}{S}_{tx}={\displaystyle \frac{\lambda R}{V_2}},\end{array}$$

(19)

where

$$\begin{array}{c}\hfill (V_1,V_2)=\left\{\begin{array}{c}(M_{ry},M_{rx}),M\ge N\hfill \\ (N_{ty},N_{tx}),M<N.\hfill \end{array}\right.\end{array}$$

3.2. Analysis of Sensitivity to Displacement

In order to investigate the sensitivity of LOS MIMO systems with planar subarrays to the deviations from the optimal planar subarray design in (19), we introduced the deviation factors, defined as the ratio of the optimal subarray separation product to the actual subarray separation product,

$$\begin{array}{c}\hfill {\eta }_1={\displaystyle \frac{\lambda R}{S_{ry}{S}_{ty}cos\beta V_1}},{\eta }_2={\displaystyle \frac{\lambda R}{S_{rx}{S}_{tx}{V}_2}},\end{array}$$

(20)

which indicate that the actual vertical (horizontal) subarray separation product is smaller than the optimal one if ${\eta }_1$ (${\eta }_2$) is larger than one.

From (2) and (4), we can see that the system spectral efficiency and EDoF depend on the eigenvalues of $\mathbf{V}$. For ease of illustration, we considered the case of $N_{ty}=N_{tx}=M_{ry}=M_{rx}=P=Q=2$, and thus the equivalent transmission channel matrix $\tilde{\mathbf{H}}$ can be expressed as

$$\begin{array}{c}\hfill \tilde{\mathbf{H}}={\mathbf{W}}^H\mathbf{H}\mathbf{F}=\left[\begin{array}{cccc}{\tilde{\mathbf{H}}}_{0,0}& {\tilde{\mathbf{H}}}_{0,1}& {\tilde{\mathbf{H}}}_{0,2}& {\tilde{\mathbf{H}}}_{0,3}\\ {\tilde{\mathbf{H}}}_{1,0}& {\tilde{\mathbf{H}}}_{1,1}& {\tilde{\mathbf{H}}}_{1,2}& {\tilde{\mathbf{H}}}_{1,3}\\ {\tilde{\mathbf{H}}}_{2,0}& {\tilde{\mathbf{H}}}_{2,1}& {\tilde{\mathbf{H}}}_{2,2}& {\tilde{\mathbf{H}}}_{2,3}\\ {\tilde{\mathbf{H}}}_{3,0}& {\tilde{\mathbf{H}}}_{3,1}& {\tilde{\mathbf{H}}}_{3,2}& {\tilde{\mathbf{H}}}_{3,3}\end{array}\right],\end{array}$$

(21)

where

$$\begin{array}{c}\hfill {\tilde{\mathbf{H}}}_{m,n}={\mathbf{w}}_m^H{\mathbf{H}}_{m,n}{\mathbf{f}}_n.\end{array}$$

When the main beam direction of the nth transmit subarray (the mth receive subarray) satisfies (${\alpha }_{nt}$,${\varphi }_{nt}$) = (${\theta }_{nt}$,${\phi }_{nt}$) ((${\alpha }_{mr}$,${\varphi }_{mr}$) = (${\theta }_{mr}$,${\phi }_{mr}$)), substituting (21) into (3) leads to

$$\begin{array}{c}\hfill \mathbf{V}=P^2{Q}^2·\left[\begin{array}{cccc}{G}_{0,0}& G_{0,1}& G_{0,2}& G_{0,3}\\ G_{1,0}& G_{1,1}& G_{1,2}& G_{1,3}\\ G_{2,0}& G_{2,1}& G_{2,2}& G_{2,3}\\ G_{3,0}& G_{3,1}& G_{3,2}& G_{3,3}\end{array}\right],\end{array}$$

(22)

where

$$\begin{array}{c}\hfill G_{n_1,n_2}=\sum _{m=0}^{M-1}{h}_{m,n_1}^H{h}_{m,n_2}=\sum _{m=0}^{M-1}exp\left(j{\displaystyle \frac{2\pi }{\lambda }}(l_{m,n_2}-l_{m,n_1})\right).\end{array}$$

Substituting (9) into (22), we can rewrite $\mathbf{V}$ as

$$\mathbf{V}=P^2{Q}^2\left[\begin{array}{cccc}{V}_1{V}_2& V_2{X}_1{Y}_2{Z}_2& V_1{X}_2{Y}_1{Z}_1& X_1{X}_2{Y}_1{Y}_2{Z}_1{Z}_2\\ V_2{X}_1^{-1}{Y}_2^{-1}{Z}_2& V_1{V}_2& X_1^{-1}{X}_2{Y}_1{Y}_2^{-1}{Z}_1{Z}_2& V_1{X}_2{Y}_1{Z}_1\\ V_1{X}_2^{-1}{Y}_1^{-1}{Z}_1& X_1{X}_2^{-1}{Y}_1^{-1}{Y}_2{Z}_1{Z}_2& V_1{V}_2& V_2{X}_1{Y}_2{Z}_2\\ X_1^{-1}{X}_2^{-1}{Y}_1^{-1}{Y}_2^{-1}{Z}_1{Z}_2& V_1{X}_2^{-1}{Y}_1^{-1}{Z}_1& V_2{X}_1^{-1}{Y}_2^{-1}{Z}_2& V_1{V}_2\end{array}\right],$$

where

$$\begin{array}{c}\hfill X_1=exp\left(j\frac{\pi }{\lambda R}·S_{tx}^2\right),X_2=exp\left(j\frac{\pi }{\lambda R}·S_{ty}^2\right)\end{array}$$

and

$$\begin{array}{c}\hfill Y_i=exp\left(-j{\displaystyle \frac{\pi }{2{\eta }_i}}\right),Z_i=\frac{sin(\pi /{\eta }_i)}{sin(\pi /\left(2{\eta }_i\right))}(i=1,2).\end{array}$$

We find that the eigenvalues of $\mathbf{V}$ are equivalent to the eigenvalues of ${\mathbf{V}}^{\prime }$, which is given by

$$\begin{array}{c}\hfill {\mathbf{V}}^{\prime }=P^2{Q}^2·\left[\begin{array}{cccc}{V}_1{V}_2& V_2{Z}_2& V_1{Z}_1& Z_1{Z}_2\\ V_2{Z}_2& V_1{V}_2& Z_1{Z}_2& V_1{Z}_1\\ V_1{Z}_1& Z_1{Z}_2& V_1{V}_2& V_2{Z}_2\\ Z_1{Z}_2& V_1{Z}_1& V_2{Z}_2& V_1{V}_2\end{array}\right].\end{array}$$

Then, the eigenvalues can be found as

$$\begin{array}{c}\hfill {\mu }_1=V_1{V}_2{P}^2{Q}^2-P^2{Q}^2(V_1{Z}_1+V_2{Z}_2)+P^2{Q}^2{Z}_1{Z}_2,\end{array}$$

(23)

$$\begin{array}{c}\hfill {\mu }_2=V_1{V}_2{P}^2{Q}^2+P^2{Q}^2(V_1{Z}_1-V_2{Z}_2)-P^2{Q}^2{Z}_1{Z}_2,\end{array}$$

(24)

$$\begin{array}{c}\hfill {\mu }_3=V_1{V}_2{P}^2{Q}^2+P^2{Q}^2(V_1{Z}_1+V_2{Z}_2)+P^2{Q}^2{Z}_1{Z}_2\end{array}$$

(25)

and

$$\begin{array}{c}\hfill {\mu }_4=V_1{V}_2{P}^2{Q}^2-P^2{Q}^2(V_1{Z}_1-V_2{Z}_2)-P^2{Q}^2{Z}_1{Z}_2.\end{array}$$

(26)

3.3. Upper and Lower Bounds of the EDoF Estimation

It is shown in (4) that the EDoF is determined by the distribution of eigenvalues of $\mathbf{V}$. The sum of the eigenvalues can be calculated as

$$\begin{array}{c}\hfill \begin{array}{c}{\displaystyle \sum _{i=1}^U}{\mu }_i=\mathrm{tr}\left({\tilde{\mathbf{H}}}^H\tilde{\mathbf{H}}\right)={\displaystyle \sum _{m=0}^{M-1}}{\displaystyle \sum _{n=0}^{N-1}}{∥{\mathbf{w}}_m^H{\mathbf{H}}_{m,n}{\mathbf{f}}_n∥}_F^2\hfill \\ \le {\displaystyle \sum _{m=0}^{M-1}}{\displaystyle \sum _{n=0}^{N-1}}{∥{\mathbf{w}}_m^H∥}_2^2{∥{\mathbf{H}}_{m,n}∥}_F^2{∥{\mathbf{f}}_n∥}_2^2=NM{P}^2{Q}^2,\hfill \end{array}\end{array}$$

where the equal sign holds only when the vector ${\mathbf{w}}_m^H$ (${\mathbf{f}}_n$) is linearly dependent on each column (row) of the subchannel ${\mathbf{H}}_{m,n}$, which corresponds to the case that all eigenvalues in (23)–(26) are equal to $V_1{V}_2{P}^2{Q}^2$ when ${\eta }_1={\eta }_2=0$ dB.

When the subarray separations at the transceiver satisfy (19), i.e., ${\tilde{\mathbf{H}}}^H\tilde{\mathbf{H}}=M{P}^2{Q}^2{\mathbf{I}}_N$, different columns or rows of the equivalent channel matrix meet the orthogonality condition, which is equivalent to U independent SISO subchannels. In this case, $rank\left(\mathbf{V}\right)=U$ and all eigenvalues of $\mathbf{V}$ are equal to $NM{P}^2{Q}^2/U$, which corresponds to the maximum EDoF given by

$$\begin{array}{c}\hfill EDo{F}_{max}=\sum _{k=1}^U\frac{\overline{\gamma }{\mu }_k}{N+\overline{\gamma }{\mu }_k}=\frac{\overline{\gamma }M{P}^2{Q}^2}{1+\overline{\gamma }{\displaystyle \frac{M{P}^2{Q}^2}{U}}}.\end{array}$$

(27)

When the separations between adjacent subarrays are zero, i.e., ${\tilde{\mathbf{H}}}^H\tilde{\mathbf{H}}=M{P}^2{Q}^2{\mathbf{1}}_N$, the RF equivalent channel is completely correlated, and becomes equivalent to a SISO channel. In this case, $rank\left(\mathbf{V}\right)=1$, and there is a unique eigenvalue ${\mu }_1=NM{P}^2{Q}^2$, which corresponds to the minimum EDoF given by

$$\begin{array}{c}\hfill EDo{F}_{min}=\frac{\overline{\gamma }{\mu }_1}{N+\overline{\gamma }{\mu }_1}=\frac{\overline{\gamma }M{P}^2{Q}^2}{1+\overline{\gamma }M{P}^2{Q}^2}.\end{array}$$

(28)

4. Quantization Codebook Design for Phase Shifters

The continuous and complete phase angle information is usually assumed to be available at the transceiver for RF phase shifter configuration. However, the phase shifters for the analog beamformer/combiner are implemented by discrete phase shifts due to the constraints on the RF hardware in a practical system, and thus the analog beamforming/combining vectors can only take specific values selected from the given quantization codebooks, as they can be easily implemented at a low cost in practical applications, though the spectral efficiency may be lower than that without quantization.

Most of the existing quantization codebooks are designed for traditional MIMO with rich scattering channels, and yet they are not applicable to LOS MIMO channels with few scatterers. According to the introduced channel model, we designed beamsteering codebooks ${\mathcal{F}}_t$ (${\mathcal{F}}_r$) to uniformly quantize the phase angle ${\alpha }_t$ (${\alpha }_r$) of the beamformer (combiner) using $N_b$ ($M_b$) bits for LOS MIMO channels, which are given by

$$\begin{array}{c}\hfill {\mathcal{F}}_t:\left\{{{\alpha }_t}_{min}+\frac{\Delta {\alpha }_t}{2^{N_b+1}},{{\alpha }_t}_{min}+\frac{3\Delta {\alpha }_t}{2^{N_b+1}},\dots ,{{\alpha }_t}_{max}-\frac{\Delta {\alpha }_t}{2^{N_b+1}}\right\}\end{array}$$

(29)

and

$$\begin{array}{c}\hfill {\mathcal{F}}_r:\left\{{{\alpha }_r}_{min}+\frac{\Delta {\alpha }_r}{2^{M_b+1}},{{\alpha }_r}_{min}+\frac{3\Delta {\alpha }_r}{2^{M_b+1}},\dots ,{{\alpha }_r}_{max}-\frac{\Delta {\alpha }_r}{2^{M_b+1}}\right\},\end{array}$$

(30)

respectively, where ${{\alpha }_t}_{min}$ and ${{\alpha }_t}_{max}$ (${{\alpha }_r}_{min}$ and ${{\alpha }_r}_{max}$) represent the minimum and maximum achievable phase angles in the beamformer (combiner) before quantization, respectively; $\Delta {\alpha }_t={{\alpha }_t}_{max}-{{\alpha }_t}_{min}$ and $\Delta {\alpha }_r={{\alpha }_r}_{max}-{{\alpha }_r}_{min}$. The quantization codebooks ${\mathcal{W}}_t$ and ${\mathcal{W}}_r$ of phase angles ${\varphi }_t$ and ${\varphi }_r$ have the same expression forms.

In the following, we characterize the effect of quantization bits on the spectral efficiency. Without a loss of generality, we assume that the elevation and azimuth angles of transmitted and received signals are evenly distributed in $\left[0,\pi \right]$ and $N_b=M_b=b$, and thus the average spectral efficiency after quantization can be expressed as

$$\begin{array}{c}\hfill SE\left(b\right)=\mathrm{E}\left\{f({\theta }_t,{\theta }_r,{\phi }_t,{\phi }_r;\mathcal{N})\right\}=\frac{1}{{\pi }^4}{\int }_0^{\pi }{\int }_0^{\pi }{\int }_0^{\pi }{\int }_0^{\pi }f({\theta }_t,{\theta }_r,{\phi }_t,{\phi }_r;\mathcal{N})d{\theta }_{t}d{\theta }_{r}d{\phi }_{t}d{\phi }_r,\end{array}$$

(31)

where $f({\theta }_t,{\theta }_r,{\phi }_t,{\phi }_r;\mathcal{N})$ denotes the system spectral efficiency given (${\theta }_t,{\theta }_r,{\phi }_t,{\phi }_r$) when the four designed codebooks $\mathcal{N}=\{{\mathcal{F}}_t,{\mathcal{F}}_r,{\mathcal{W}}_t,{\mathcal{W}}_r\}$ are used. Note that the quantized phase angles in the codebooks are selected such that an estimated phase angle has the closest distance among the quantized ones in their respective codebook.

For ease of illustration, we assume that the phase shifters for phase angles ${\varphi }_t$ and ${\varphi }_r$ are implemented by continuous phase shifts, i.e., ${\varphi }_t={\phi }_t$ and ${\varphi }_r={\phi }_r$. The minimum and maximum achievable phase angles are assumed to be 0 and $\pi$ for ${\alpha }_t$ and ${\alpha }_r$, respectively, and the corresponding codebook is $\left[{\displaystyle \frac{\pi }{2^{b+1}}},{\displaystyle \frac{3\pi }{2^{b+1}}},\dots ,\pi -{\displaystyle \frac{\pi }{2^{b+1}}}\right]$ from (29) and (30). Therefore, for a $2\times 2$ subarray’s pure LOS channel with ${\eta }_1={\eta }_2=0$ dB, (31) becomes

$$\begin{array}{cc}\hfill SE\left(b\right)=& {\displaystyle \frac{1}{{\pi }^2}}{\int }_0^{\pi }{\int }_0^{\pi }f({\theta }_t,{\theta }_r;{\mathcal{F}}_t,{\mathcal{F}}_r)d{\theta }_{t}d{\theta }_r\hfill \\ \hfill =& \frac{U}{{\pi }^2}{\int }_0^{\pi }{\int }_0^{\pi }{log}_2\left(1+\frac{\overline{\gamma }}{N}·\mu ({\theta }_t,{\theta }_r;{\mathcal{F}}_t,{\mathcal{F}}_r)\right)d{\theta }_{t}d{\theta }_r\hfill \\ \hfill \stackrel{\left(a\right)}{\approx }& {\displaystyle \frac{U}{{\pi }^2}}\sum _{c_1=0}^{2^b-1}\sum _{c_2=0}^{2^b-1}{\int }_{\frac{c_1\pi }{2^b}}^{\frac{(c_1+1)\pi }{2^b}}{\int }_{\frac{c_2\pi }{2^b}}^{\frac{(c_2+1)\pi }{2^b}}\left[\frac{k_b\overline{\gamma }}{N}{f}_1\left({\theta }_t;\frac{(2c_1+1)\pi }{2^{b+1}}\right)\right.\hfill \\ \hfill & \left.·f_2\left({\theta }_r;\frac{(2c_2+1)\pi }{2^{b+1}}\right)+a_b\right]d{\theta }_{t}d{\theta }_r\hfill \\ \hfill =& U·a_b+\frac{U}{{\pi }^2}·\frac{k_b\overline{\gamma }}{N}\sum _{c_1=0}^{2^b-1}\sum _{c_2=0}^{2^b-1}\hfill \\ \hfill & \left[F_1\left(\frac{(c_1+1)\pi }{2^b};\frac{(2c_1+1)\pi }{2^{b+1}}\right)-F_1\left(\frac{c_1\pi }{2^b};\frac{(2c_1+1)\pi }{2^{b+1}}\right)\right]\hfill \\ \hfill & \left[F_2\left(\frac{(c_2+1)\pi }{2^b};\frac{(2c_2+1)\pi }{2^{b+1}}\right)-F_2\left(\frac{c_2\pi }{2^b};\frac{(2c_2+1)\pi }{2^{b+1}}\right)\right],\hfill \end{array}$$

(32)

where the specific expressions of $f_1\left({\theta }_t;\frac{(2c_1+1)\pi }{2^{b+1}}\right)$ and $f_2\left({\theta }_r;\frac{(2c_2+1)\pi }{2^{b+1}}\right)$ are shown as

$$\begin{array}{c}\hfill f_1\left({\theta }_t;\frac{(2c_1+1)\pi }{2^{b+1}}\right)={\left|\sum _{p=0}^{P^2-1}exp\left(j{d}^{\prime }\left(sin{\theta }_t-sin\left(\frac{(2c_1+1)\pi }{2^{b+1}}\right)\right)(p^{x}cos{\phi }_t+p^{y}sin{\phi }_t)\right)\right|}^2,\\ \hfill f_2\left({\theta }_r;\frac{(2c_2+1)\pi }{2^{b+1}}\right)={\left|\sum _{q=0}^{Q^2-1}exp\left(j{d}^{\prime }\left(sin{\theta }_r-sin\left(\frac{(2c_2+1)\pi }{2^{b+1}}\right)\right)(q^{x}cos{\phi }_r+q^{y}sin{\phi }_r)\right)\right|}^2.\end{array}$$

Their integral primitive functions are represented as $F_1\left({\theta }_t;\frac{(2c_1+1)\pi }{2^{b+1}}\right)$ and $F_2\left({\theta }_r;\frac{(2c_2+1)\pi }{2^{b+1}}\right)$, respectively. In (32), the approximation $\left(a\right)$ holds because ${log}_2\left(1+\frac{\overline{\gamma }}{N}\mu \right)$ can be approximated as linear functions of $\mu$, i.e., $k_b·\frac{\overline{\gamma }}{N}\mu +a_b$ for different numbers of quantization bits when $\frac{\overline{\gamma }}{N}·\mu \gg 1$. The constants $k_b$ and $a_b$ can be calculated offline given other parameters. For a given set of parameters, the theoretical results of (32) can be obtained by high-precision numerical integration.

Figure 3 compares the spectral efficiency $SE\left(b\right)$ among different quantization strategies of using the designed codebooks, without quantization and with the quantization scheme in [24], where $2\times 2$ subarrays (each with $2\times 2$ or $4\times 4$ antennas) and LOS channels are considered. It is shown that $SE\left(2\right)\approx 28.60$ bits/s/Hz, $SE\left(3\right)\approx 29.15$ bits/s/Hz and $SE\left(4\right)\approx 29.30$ bits/s/Hz, which becomes increasingly closer to the spectral efficiency without quantization, $29.32$ bits/s/Hz in (2), with the increasing quantization bits. Targeting a loss of spectral efficiency within $5\%$, two quantization bits are needed for each subarray with $2\times 2$ antennas, whereas three quantization bits are required for each subarray with $4\times 4$ antennas. This indicates that the quantization bits for each phase angle are required to increase by one bit when the dimension of arrays is doubled. The number of bits required to properly quantize the phase angles increases with the increase in array size, as larger arrays produce narrower beams and require finer steering. The results in Figure 3 show that the designed codebooks are superior to that in [24] in terms of spectral efficiency using the same number of quantization bits. The performance under different Ricean factors K is also provided for comparison.

5. Numerical and Simulation Results

In this section, we present the numerical and simulation results to evaluate the LOS MIMO system performance using hybrid arrays with planar subarrays at the transceiver. Unless stated otherwise, we considered hybrid arrays with $N_{ty}=N_{tx}=M_{ry}=M_{rx}=P=Q=2$ and the carrier frequency of 220 GHz. We chose $S_{tx}=S_{ty}=S_{rx}=S_{ry}=\sqrt{{\displaystyle \frac{\lambda R}{2}}}$ and $d={\displaystyle \frac{\lambda }{2}}$ as the subarray separation for a practical array placement to achieve the optimal performance at $\beta =0^{\circ }$ and $R=50$ m. The average SNR after analog beamforming combining, $\overline{\gamma }$, was set to be 10 dB in the simulation.

Figure 4 shows the eigenvalues $\{{\mu }_i{\}}_{i=1,\dots ,4}$ versus the deviation factor $\eta$, where ${\eta }_1={\eta }_2=\eta$ is assumed. There are only three eigenvalue curves in the figure due to ${\mu }_2={\mu }_4$. It is found from (20) that the optimal performance is achieved when $\eta =0$ dB, which corresponds to the smallest subarray separation product in practical applications. Substituting $\eta =0$ dB into (23)–(26), we have $\{{\mu }_i{\}}_{i=1,\dots ,4}=64$, which is consistent with the simulation results. When $\eta >0$ dB, the LOS MIMO channels will approach the statistical characteristics of SISO channels. The associated eigenvalue ${\mu }_3=16P^2{Q}^2=256$, and the others become zero when $\eta \to \infty$. When $\eta <0$ dB, the optimal performance can be achieved a few times corresponding to $\{{\mu }_i{\}}_{i=1,\dots ,4}=64$ due to the periodicity of the trigonometric function in (17).

Figure 5 shows the cumulative distribution function (CDF) of EDoF for traditional array (TRA) and planar subarray (PLA) structures considering the optimal design in the analog domain and design mismatch, where traditional arrays are assumed to be the $2\times 2$ antennas at the transceiver, respectively. It is observed that the two array structures have a similar trend of CDFs under the same conditions, but planar subarrays are always superior to traditional arrays in terms of EDoF. When $K=-10$ dB, i.e., the channel approaches Rayleigh fading, traditional arrays exhibit a close performance under different $\eta$, whereas there is an obvious difference for planar subarrays, which means that it is sensitive to $\eta$. When $K=15$ dB, i.e., the channel approaches a pure LOS condition, the EDoF becomes more sensitive to $\eta$. It is seen that LOS channels with an optimal subarray separation product are superior in term of EDoF compared to MIMO systems based on independent identically distributed Rayleigh channels.

In Figure 6, the EDoFs are plotted as functions of $\eta$ with different Ricean factors K. It is shown from this figure that the EDoF becomes increasingly dependent on $\eta$ with the increase in K. It is also interesting to note that the EDoF using planar subarrays is almost independent of K when $\eta \approx 7$ dB, whereas $\eta \approx 4$ dB for traditional arrays. Under the same conditions, using planar subarrays can achieve a higher EDoF than traditional arrays. This figure reveals that, for the traditional array structure, the best performance for $K=+\infty$ can be obtained only when the corresponding parameters meet the Rayleigh distance criterion. The EDoF will be degraded once $\eta$ deviates from the optimal design in the analog domain. However, for the planar subarray structure, it becomes less sensitive when the parameters deviate from the optimal design in the analog domain within a certain range. This means that the planar subarray structure is more robust to the design mismatch. The minimum and maximum EDoFs in (27) and (28) were calculated as $EDo{F}_{max}\approx 3.9752$ and $EDo{F}_{min}\approx 0.9984$, respectively. For traditional arrays, $EDo{F}_{max}\approx 3.6364$ and $EDo{F}_{min}\approx 0.9756$. The above calculations are consistent with the simulation results in Figure 6. Figure 7 shows the 3D graphics of the system spectral efficiency as the function of ${\eta }_1$ and ${\eta }_2$ in a pure LOS MIMO channel.

Figure 8 investigates how sensitive the EDoF is to the communication distance R for different carrier frequencies. It can be seen that the EDoF decreases with increasing R, but it falls more slowly using planar subarrays than traditional arrays. The EDoF of the former is always higher than that of the latter. These indicate that the use of planar subarrays not only reduces the sensitivity of EDoF to the change in R, but also achieves a higher EDoF under the same conditions compared to traditional arrays in LOS MIMO systems. We can also see that the system EDoF using planar subarrays is also less sensitive to the change in carrier frequency compared to using traditional arrays.

Figure 9 shows the spectral efficiency as a function of phase angles ${\alpha }_t$ and ${\alpha }_r$ in the analog beamformer and combiner, respectively, where the phase angles ${\varphi }_t$ and ${\varphi }_r$ are assumed to be equal to $\pi /3$ for simplicity. We also assume the elevation and azimuth angles of the signals, ${\theta }_{nt}={\theta }_{mr}=\pi /6$ and ${\phi }_{nt}={\phi }_{mr}=\pi /3$, $\forall n$, m. Through a comprehensive analysis of the above two figures, it can be observed that the system spectral efficiency achieves the maximum value when ${\alpha }_t={\alpha }_r=\pi /6$, which corresponds to the main beam directions of all subarrays directed at the directions of incoming signals. The large mismatch of phase angles will cause a serious loss in the system spectral efficiency.

6. Conclusions

Based on the introduced joint plane-wave and spherical-wave 3D channel model for hybrid arrays with planar subarrays, we investigated the spatial multiplexing performance of mmWave LOS MIMO systems and derived the optimal subarray separation products in the vertical and horizontal directions with respect to maximizing the spectral efficiency and EDoF. It is shown that the subarray separation products can be represented as a function of the communication distance, carrier wavelength, the number of subarrays and the angles of the array placement. Compared to traditional arrays, a planar subarray structure can not only achieve a higher EDoF but can also be more robust to the change in the communication distance and the NLOS components. Therefore, using the planar subarray structure can remove the limitation that the optimal performance is achieved only when the Rayleigh distance criterion is satisfied for traditional arrays. It is also found that the influence of the carrier frequency on the system EDoF in planar subarrays is lower than that in traditional arrays. With the designed quantization codebooks, it was found that, when the dimension of arrays is doubled, the number of quantization bits of each phase angle needs to be increased by one to maintain the performance. In future work, the considered system models can be extended to a multi-user case for more general applications by exploiting the user separation in the angular domain. How to further develop the joint optimal design approaches for analog–digital hybrid beamforming, e.g., exploiting the iterative method in [37] for multi-user LOS MIMO systems, will be an interesting future research topic.