An Off-Grid Turbo Channel Estimation Algorithm for Millimeter Wave Communications

Abstract

The bandwidth shortage has motivated the exploration of the millimeter wave (mmWave) frequency spectrum for future communication networks. To compensate for the severe propagation attenuation in the mmWave band, massive antenna arrays can be adopted at both the transmitter and receiver to provide large array gains via directional beamforming. To achieve such array gains, channel estimation (CE) with high resolution and low latency is of great importance for mmWave communications. However, classic super-resolution subspace CE methods such as multiple signal classification (MUSIC) and estimation of signal parameters via rotation invariant technique (ESPRIT) cannot be applied here due to RF chain constraints. In this paper, an enhanced CE algorithm is developed for the off-grid problem when quantizing the angles of mmWave channel in the spatial domain where off-grid problem refers to the scenario that angles do not lie on the quantization grids with high probability, and it results in power leakage and severe reduction of the CE performance. A new model is first proposed to formulate the off-grid problem. The new model divides the continuously-distributed angle into a quantized discrete grid part, referred to as the integral grid angle, and an offset part, termed fractional off-grid angle. Accordingly, an iterative off-grid turbo CE (IOTCE) algorithm is proposed to renew and upgrade the CE between the integral grid part and the fractional off-grid part under the Turbo principle. By fully exploiting the sparse structure of mmWave channels, the integral grid part is estimated by a soft-decoding based compressed sensing (CS) method called improved turbo compressed channel sensing (ITCCS). It iteratively updates the soft information between the linear minimum mean square error (LMMSE) estimator and the sparsity combiner. Monte Carlo simulations are presented to evaluate the performance of the proposed method, and the results show that it enhances the angle detection resolution greatly.

1. Introduction

Thanks to the large bandwidth available at millimeter wave (mmWave) frequencies, mmWave communication technology has become a promising technology to meet the experientially increasing demands of future wireless networks [1]. The small wavelength at the mmWave band enables the integration of a massive number of antennas at both the transmitter and receiver. The directional beamforming technique can be used to achieve a sufficient link margin to compensate for the severe propagation attenuation in mmWave bands [2,3]. Since the channel state information, especially the angle of departure (AoD) and angle of arrival (AoA), are essential for the beamforming and coherent detection, CE with high resolution and efficiency is one of the key requirements in mmWave systems. Due to the high cost and high power consumption of mmWave radio frequency (RF) devices, only a very limited number of RF chains can be integrated in the mmWave systems, which results in much less RF chains than antennas. Therefore, classic angle estimation methods with super-resolution, such as multiple signal classification (MUSIC) and estimation of signal parameters via rotation invariant technique (ESPRIT)-based algorithms [4,5], cannot be directly applied in mmWave systems because these methods require the channel observations at each receive antenna in the digital domain. However, in the mmWave systems, the received signals at each antenna are firstly combined in the analog domain and thus no direct output information from each antenna is available for these classic angle estimation methods. Therefore, new CE algorithms with high-resolution should be developed and tailored to mmWave communications.

Recently, many research efforts have been devoted to the CE for the mmWave systems [6,7,8,9,10]. As adopted in IEEE 802.15.3c standard [6], a polling mechanism is employed to select the best beam vector pair from the known codebooks with p beams at the transmitter and q beams at the receiver. This method consumes $pq$ time slots to achieve angle resolution $O(1/p)$ and $O(1/q)$. In [7], an adaptive compressive sensing (ACS) method has been developed, which iteratively bisections the beam space and tries to find the grid that the path falls in via space match filtering. It needs $2L^{3}lo{g}_2(p/L)+2L^{3}lo{g}_2(q/L)$ time slots to achieve the resolution $O(1/p)$ at the transmitter and $O(1/q)$ at the receiver for an mmWave channel with L paths. This large time slot payload is unacceptable in practical mobile communication systems with massive antenna arrays, especially in the outdoor wireless coverage scenarios. So far, most existing mmWave CE methods, such as [8,9,10], are based on the on-grid channel model, i.e., all path angles are assumed to be located at the quantization grids exactly, which is not the case in practical systems because the angles are usually continuously distributed. When path angles do not lie on the quantization grids, which we refer to as the “off-grid” channel model, a power leakage effect happens and results in the degradation of CE performance [11]. Since the product of the number of simultaneously supported multi-users and the number of the multi-streams depends on the RF chains, it is preferable that higher angle resolution is achieved by advanced channel estimation methods to separate densely distributed users such that spatial multiplexing methods can be employed to support more users [12]. In this ”off-grid” case, the gap to the theoretic lower bound is pretty large, and more antennas are required to enhance the resolution of angle estimation at the cost of more hardware and antennas [13]. Therefore, a new method is necessary to enhance the angle detection performance. So far, few works concentrate on the off-grid problem in the mmWave systems with RF constraints. In [14], the continuous angle estimation was formulated as a sparse regularization problem by taking into account the noises in both the observations and the dictionary by modeling the grid points as noisy, which is hard to be applied in CE because the spectral norm of the noise is much bigger than that of the dictionary. In [15], a continuous basis-based compressed sensing (CS) method was proposed to solve the reconstruction problem in sensing applications where the atoms of the dictionary are used to describe a continuous field, such as frequency and angle. However, the first order approximation of Taylor expansion for the array steering vector in the multi-antenna systems is hard to formulate.

In this paper, we study the off-grid problem and propose an enhanced method to improve the resolution of angle estimation in mmWave systems with massive antenna arrays and RF chain constraints. Firstly, we formulate the signal model to describe the off-grid problem in the angle quantization of the mmWave channel, which decomposes each continuously-distributed angle into the integral grid part and fractional off-grid part. Accordingly, an iterative off-grid turbo CE (IOTCE) algorithm is developed to renew and upgrade the CE between the integral grid and fractional off-grid angle information under the Turbo principle. By fully exploiting the sparse structure of mmWave channels, a soft-decoding based CS method, termed improved turbo compressed channel sensing (ITCCS), is developed to estimate the integral grid angle information, which iteratively updates the soft information between the linear minimum mean square error (LMMSE) estimator and the sparsity combiner. Simulation results show that the proposed CE method can achieve a higher resolution than the existing ones.

The rest of the paper is organized as follows. The mmWave system model is introduced in Section 2. Then, a new off-grid formulation is specified in Section 3 and the proposed IOTCE algorithm is detailed in Section 4. Section 5 presents the simulation results and Section 6 concludes the paper.

Notations: Fonts $\mathit{A}$, $\mathit{a}$, a and $\mathbb{A}$ denote a matrix, a column vector, a scalar and a set, respectively. ${\mathit{A}}^T$ and ${\mathit{A}}^H$ are the transpose and Hermitian of $\mathit{A}$. ${\mathit{1}}_N\in {\mathbb{C}}^{N\times 1}$ is an all-ones vector and $\mathit{I}$ is an identity matrix. ${\mathit{e}}_N(i)\in {\mathbb{C}}^{N\times 1}$ is a unit vector with 1 at its i-th entry and 0 elsewhere. The operator ⊗ are used to denote the Kronecker product. $\text{Vect}(\mathit{A})$ is the matrix vectorization operator that transforms a matrix into a vector by stacking its column vectors, and $\text{Diag}(\mathit{a})$ stands for a diagonal matrix with the entries of $\mathit{a}$ on its diagonal. ${[\mathit{A}]}_{\mathcal{R},\mathcal{P}}$ consists of row vectors and column vectors of the matrix $\mathit{A}$ whose indices contained in the set $\mathcal{R}$ and $\mathcal{P}$, respectively. $\mathbb{E}(·)$ represents the expectation. $\delta (·)$ is the Dirac delta function and $I(·)$ is the indicator function.

2. System Model

According to [16], typical mmWave channels feature much stronger line-of-sight (LoS) path than non-LoS (NLoS) paths, and the numerical results in [17] show that approximating the sparse mmWave channel with only one LoS path results in a little capacity loss when employing the steerable directional antennas. Thus, single-path mmWave channel model is used in this paper. When the mmWave channel cannot be approximated as a single-path channel, the proposed algorithm cannot be applied directly. New algorithms need to be developed to deal with independently but not identically distributed multi-paths. Consider a downlink mmWave system consisting of a base station (BS) and a mobile station (MS), where the BS is configured with $N_B$ antennas and $K_B$ RF chains with $K_B\ll N_B$ while the MS is configured with $N_M$ antennas and $K_M$ RF chains with $K_M\ll N_M$, as shown in Figure 1. The BS applies the beamforming matrix ${\mathit{W}}_B\in {\mathbb{C}}^{N_B\times K_B}$ and the MS employs the matrix ${\mathit{W}}_M\in {\mathbb{C}}^{N_M\times K_M}$ to combine the received signal $\mathit{y}\in {\mathbb{C}}^{K_M\times 1}$. The received signal at the MS can be written as:

$$\begin{array}{c}\hfill \mathit{y}={\mathit{W}}_M^T\mathit{H}{\mathit{W}}_B\mathit{x}+\mathit{z}\end{array}$$

(1)

where $\mathit{H}\in {\mathbb{C}}^{N_M\times N_B}$ is the channel matrix, $\mathit{x}\in {\mathbb{C}}^{K_B\times 1}$ represents the transmitted pilot signal, and $\mathit{z}\in {\mathbb{C}}^{K_N\times 1}$ is the Gaussian noise with $\mathbb{E}({\mathit{zz}}^H)={\sigma }^2\mathit{I}$.

In this paper, we also assume:

• uniform linear arrays (ULAs) with half-wavelength spacing are deployed at both the BS and MS;
• far-field scattering and block-fading are held, which means the signal waves arrive at different antennas with the same fading amplitudes but distinct phases and the channel fadings are kept constant during the CE precedure.

Under these assumptions, the single-path channel between the BS and MS can be expressed as:

$$\begin{array}{c}\hfill \mathit{H}=\beta {\mathit{a}}_M({\theta }_M){\mathit{a}}_B^T({\theta }_B)\end{array}$$

(2)

where β is the fading weight, ${\theta }_B\in [-\frac{\pi }{2},\frac{\pi }{2})$, and ${\theta }_M\in [-\frac{\pi }{2},\frac{\pi }{2})$ are, respectively, AoD and AoA. ${\mathit{a}}_B$ and ${\mathit{a}}_M$ are the array steering vector at the BS and the combining vector at the MS, respectively. Denoting $\frac{sin{\theta }_B}{2}$ by ${\varphi }_B\in [-\frac{1}{2},\frac{1}{2})$ and $\frac{sin{\theta }_M}{2}$ by ${\varphi }_M\in [-\frac{1}{2},\frac{1}{2})$, ${\mathit{a}}_B$ and ${\mathit{a}}_M$ can be written as:

$$\begin{array}{c}\hfill {\mathit{a}}_B({\varphi }_B)=\frac{1}{\sqrt{N_B}}{\left[1e^{j2\pi {\varphi }_B}\cdots e^{j2\pi (N_B-1){\varphi }_B}\right]}^T\end{array}$$

(3)

$$\begin{array}{c}\hfill {\mathit{a}}_M({\varphi }_M)=\frac{1}{\sqrt{N_M}}{\left[1e^{j2\pi {\varphi }_M}\cdots e^{j2\pi (N_M-1){\varphi }_M}\right]}^T\end{array}$$

(4)

The $(m,n)$-th element of the channel can be rewritten as:

$$\begin{array}{cc}\hfill {[\mathit{H}]}_{m,n}& =\beta e^{j2\pi (m-1){\varphi }_M}{e}^{j2\pi (n-1){\varphi }_B}\hfill \\ & ={\int }_{-\frac{1}{2}}^{\frac{1}{2}}{\int }_{-\frac{1}{2}}^{\frac{1}{2}}h({\tilde{\varphi }}_M,{\tilde{\varphi }}_B)e^{j2\pi (m-1){\tilde{\varphi }}_M}{e}^{j2\pi (n-1){\tilde{\varphi }}_B}d{\tilde{\varphi }}_{M}d{\tilde{\varphi }}_B\hfill \end{array}$$

(5)

where $h({\tilde{\varphi }}_M,{\tilde{\varphi }}_B)=\beta \delta ({\tilde{\varphi }}_M-{\varphi }_M)\delta ({\tilde{\varphi }}_B-{\varphi }_B)$ is the channel impulse response (CIR). By discretizing Equation (5) at the angle period $[-\frac{1}{2},\frac{1}{2})$,

$$\begin{array}{c}\hfill {[\mathit{H}]}_{m,n}(k,i)={\displaystyle \sum _{k=-N_M/2}^{N_M/2-1}\sum _{i=-N_B/2}^{N_B/2-1}}h(k/N_M,i/N_B)e^{j2\pi (m-1)(k+{\delta }_M)/N_M}{e}^{j2\pi (n-1)(i+{\delta }_B)/N_B}\end{array}$$

(6)

where ${\delta }_M$ and ${\delta }_B$ are the discretization errors corresponding to the ${\varphi }_M$ and ${\varphi }_B$, respectively. The on-grid algorithms, such as [6,7,8,9,10], can estimate the AoA/AoD effectively, by ignoring the discretization errors and assuming that the AoA/AoD are taken from a uniform grid of N points, i.e., $\varphi \in \{-\frac{1}{2},-\frac{1}{2}+\frac{1}{N},\dots ,\frac{1}{2}-\frac{1}{N}\},$ where N stands for the number of antennas and ϕ stands for the angle. In practice, the number of antennas is limited and the AoA/AoD are actually continuous, which causes the leakage of power, and, in turn, leads to the degradation of CE performance.

3. Off-Grid Channel Formulation

In order to suppress the negative effect caused by the off-grid problem, a specific and operable virtual channel representation model is introduced. From this virtual model, an enhanced off-grid angle estimation model is developed.

The continuously-distributed AoA can be written as:

$$\begin{array}{c}\hfill {\varphi }_M=-\frac{1}{2}+\frac{{\varphi }_{M,k}}{N_M}+\frac{{\delta }_M}{N_M}\end{array}$$

(7)

where ${\varphi }_{M,k}=k-1$ is the nearest k-th ($k=1,2,\dots ,N_M$) discrete quantization grid of ${\varphi }_M$, which is referred to as the integral grid angle, and $\left|{\delta }_M\right|<\frac{1}{2}$ stands for the deviation between ${\varphi }_M$ and ${\varphi }_{M,k}$, termed fractional off-grid angle. Similarly, the discrete form of the AoD is formulated as:

$$\begin{array}{c}\hfill {\varphi }_B=-\frac{1}{2}+\frac{{\varphi }_{B,i}}{N_B}+\frac{{\delta }_B}{N_B}\end{array}$$

(8)

where ${\varphi }_{B,i}=i-1$ ($i=1,2,\dots ,N_B$) and $\left|{\delta }_B\right|<\frac{1}{2}$ are, respectively, the integral grid part and fractional off-grid part of the angle. Then, the fractional off-grid angle vectors ${\mathit{d}}_M\in {\mathbb{C}}^{N_M\times 1}$ and ${\mathit{d}}_B\in {\mathbb{C}}^{N_B\times 1}$ can be rewritten as:

$$\begin{array}{c}\hfill {\mathit{d}}_M({\delta }_M)={\left[1e^{j2\pi {\delta }_M/N_M}\cdots e^{j2\pi (N_M-1){\delta }_M/N_M}\right]}^T\end{array}$$

(9)

$$\begin{array}{c}\hfill {\mathit{d}}_B({\delta }_B)={\left[1e^{j2\pi {\delta }_B/N_B}\cdots e^{j2\pi (N_B-1){\delta }_B/N_B}\right]}^T\end{array}$$

(10)

Accordingly, the channel matrix in Equation (2) can be reformulated as:

$$\begin{array}{cc}\hfill \mathit{H}& =\beta \text{Diag}{({\mathit{d}}_M({\delta }_M))}^T{\mathit{a}}_M(-\frac{1}{2}+\frac{{\varphi }_{M,k}}{N_M}){\mathit{a}}_B^T(-\frac{1}{2}+\frac{{\varphi }_{B,i}}{N_B})\text{Diag}({\mathit{d}}_B({\delta }_B))\hfill \\ & ={\mathit{D}}_M^T{\mathit{H}}_D{\mathit{D}}_B\hfill \end{array}$$

(11)

where ${\mathit{H}}_D=\beta {\mathit{a}}_M(-\frac{1}{2}+\frac{{\varphi }_{M,k}}{N_M}){\mathit{a}}_B^T(-\frac{1}{2}+\frac{{\varphi }_{B,i}}{N_B})$ represents the channel matrix only containing the integral grid part, and the diagonal matrices ${\mathit{D}}_M=\text{Diag}({\mathit{d}}_M({\delta }_M))\in {\mathbb{C}}^{N_M\times N_M}$ and ${\mathit{D}}_B=\text{Diag}({\mathit{d}}_B({\delta }_B))\in {\mathbb{C}}^{N_B\times N_B}$ represent the fractional off-grid angle matrixes. From the virtual channel model in the [18], ${\mathit{H}}_D$ can be rewritten as:

$$\begin{array}{c}\hfill {\mathit{H}}_D={\mathit{F}}_M^T{\mathit{GF}}_B\end{array}$$

(12)

where ${\mathit{F}}_B\in {\mathbb{C}}^{N_B\times N_B}$ is the discrete Fourier transform (DFT) matrix whose $(m_B,n_B)$-th entry is $\frac{1}{\sqrt{N_B}}{e}^{j2\pi m_B\frac{n_B}{N_B}}$, $0\le m_B,n_B\le N_B-1$, and ${\mathit{F}}_M\in {\mathbb{C}}^{N_M\times N_M}$ is also a DFT matrix whose $(m_M,n_M)$-th entry is $\frac{1}{\sqrt{N_M}}{e}^{j2\pi m_M\frac{n_M}{N_M}}$, $0\le m_M,n_M\le N_M-1$. $\mathit{G}\in {\mathbb{C}}^{N_M\times N_B}$ is referred to as the integral virtual channel matrix in angle domain with only one non-zero element, and ${[\mathit{G}]}_{k,i}$ denotes the nearest integral grid point to the actual angle. The row and column labels of the non-zero element, k and i, are connected to the ${\varphi }_{M,k}$ and ${\varphi }_{B,i}$ via

$$\begin{array}{c}\hfill {\varphi }_{M,k}=k-1\end{array}$$

(13)

$$\begin{array}{c}\hfill {\varphi }_{B,i}=i-1\end{array}$$

(14)

Then, the virtual representation model of the mmWave channel with continuously-distributed angle can be written as:

$$\begin{array}{c}\hfill \mathit{H}={\mathit{D}}_M^T{\mathit{F}}_M^T\mathit{G}{\mathit{F}}_B{\mathit{D}}_B\end{array}$$

(15)

where ${\mathit{D}}_M$ and ${\mathit{D}}_B$ are unitary matrices. It is interesting that the channel matrix $\mathit{H}$ can be regarded as a two-dimensional (2D) orthogonal transform from the spatial domain to the beamforming domain. More specifically, the 2D orthogonal transform becomes the 2D DFT when the AoA/AoD locate at the quantization grids exactly.

4. The Off-Grid Turbo Channel Estimation Algorithm

In order to facilitate the operation of the proposed CE algorithm, the codebook designed in [17] is employed to divide the AoA/AoD estimation into two steps to estimate AoA and AoD separately at the BS and MS in adjacent time slots. Moreover, it can also be adopted here to estimate the fractional off-grid parts of AoA and AoD. Without loss of generality, the codebook design for AoA estimation at the MS is introduced as an example, and the same process can be employed at the BS to estimate the AoD. The process of training mode design is summarized as follows:

• Training mode at the transmitter. The training signal is designed to hold the form of $\mathit{x}={\mathit{1}}_{K_B}$, and only one column of the beamforming matrix ${\mathit{W}}_B$ is chosen as ${[{\mathit{W}}_B]}_{:,n_o}={\mathit{e}}_{N_B}(i_o)$, $n_o\in \{1,2,\cdots ,K_B\},i_o\in \{1,2,\cdots ,N_B\}$ while all other columns are zero vectors, where ${[{\mathit{W}}_B]}_{:,n_o}$ denotes the $n_o$-th column vector of matrix ${\mathit{W}}_B$. The training signals $\mathit{x}$ are transmitted from $K_B$ RF chains and beamformed with the same ${\mathit{W}}_B$ in the successive R time slots.
• Training mode at the receiver. The combining matrix at the r-th time slot, ${\mathit{W}}_M^r,r=1,2,\cdots ,R$, is designed in a way that its n-th column vector is ${[{\mathit{W}}_M^r]}_{:,n}={\mathit{e}}_{N_M}(i_n^r),n=1,2,\cdots ,K_M,$ where $\{i_1^r,i_2^r,\cdots ,i_{K_M}^r\}\triangleq {\mathcal{P}}_M^r\subset {\mathcal{R}}_M^r,$ with ${\mathcal{P}}_M^r$ being randomly chosen from ${\mathcal{R}}_M^r$ and ${\mathcal{R}}_M^1\triangleq \{1,2,\cdots ,N_M\}$, ${\mathcal{R}}_M^2=\{1,2,\cdots ,N_M\}\setminus {\mathcal{P}}_M^1,\cdots ,{\mathcal{R}}_M^r=\{1,2,\cdots ,N_M\}\setminus \{{\mathcal{P}}_M^1\cup \cdots \cup {\mathcal{P}}_M^{r-1}\}$.

In this way, the received signal at the MS can be written as:

$$\begin{array}{c}\hfill {\mathit{y}}_r={({\mathit{W}}_M^r)}^T{\mathit{D}}_M^T{\mathit{F}}_M^T{\mathit{GF}}_B{\mathit{D}}_B{\mathit{W}}_B\mathit{x}+\mathit{z}\end{array}$$

(16)

Stacking ${\mathit{y}}_r$ into a vector, we have

$$\begin{array}{cc}\hfill \mathit{y}& =[{\mathit{y}}_1^T{\mathit{y}}_2^T\cdots {\mathit{y}}_R^T]\hfill \\ & =\mathit{W}\mathit{D}\mathit{F}\mathit{g}+\mathit{z}\hfill \end{array}$$

(17)

where $\mathit{W}\triangleq {\left[{\mathit{W}}_M^1{\mathit{W}}_M^2\cdots {\mathit{W}}_{MK}^R\right]}^T\in {\mathbb{C}}^{RK\times N},$ with N and K being the number of antennas and RF chains, respectively. $\mathit{F}\in {\mathbb{C}}^{N\times N}$ is a DFT matrix, and $\mathit{g}\in {\mathbb{C}}^{N\times 1}$ is the integral virtual channel vector only containing the AoA of the LoS path seen by the MS. $\mathit{D}\in {\mathbb{C}}^{N\times N}$ is the fractional off-grid angle matrix corresponding to the AoA. Apparently, $\mathit{W}$ is a permutation matrix created by choosing K rows from an identity matrix $\mathit{I}$.

From Equation (17), the CE is accomplished by estimating the integral virtual channel, which, in turn, defines integral grid angle, and the fractional off-grid angle matrix, which stands for the fractional off-grid angle. As shown in Figure 2, an iterative off-grid turbo CE (IOTCE) algorithm is developed, which consists of two components: the integral grid angle estimator and the fractional off-grid angle estimator.

4.1. The Integral Grid Angle Estimator

Equation (17) degenerates into a CS recovery algorithm with $\mathit{W}$ being the sampling matrix and $\mathit{D}\times \mathit{F}$ being the transform matrix when the fractional off-grid angle δ is given. As the application of the TCCS algorithm in [19], the ITCCS algorithm is adopted to iteratively estimate the integral virtual channel vector g from the Equation (17). $\mathit{g}\in {\mathbb{C}}^{N\times 1}$ is a sparse signal to be estimated with L non-zero elements, following the Bernoulli–Gaussian distribution with $\lambda =\frac{L}{N}$ being the sparsity.

Let $\mathit{f}\triangleq \mathit{Fg}$ and $\mathit{q}\triangleq \mathit{D}\mathit{f}$, and Equation (17) can be rewritten as:

$$\begin{array}{c}\hfill \mathit{y}=\mathit{W}\mathit{q}+\mathit{z}\end{array}$$

(18)

As illustrated in Figure 3, the ITCCS algorithm consists of two modules: module A, the LMMSE estimator, and module B, the sparsity combiner. The LMMSE estimator provides an estimate of $\mathit{q}$ by the linear minimum mean square estimation (LMMSE) method, from which an estimate of $\mathit{g}$ can be obtained through constraints. Using the probability density estimation method, the sparsity combiner refines the estimate by exploiting the Bernoulli–Gaussian distribution. The specific realization of the algorithm can be seen in [19].

Therefore, by fixing δ and using the constraints Equations (9) and (10), we get the corresponding fractional off-grid angle vector $\mathit{d}(\delta )$ and matrix $\mathit{D}$, to estimate the integral virtual channel vector $\mathit{g}$ at the sparsity $\lambda =\frac{L}{N}$ ($L=1$) using the ITCCS algorithm, where the value of δ is zero at the first iteration and is fed back from the fractional off-grid angle estimator at the iteration. For the purposes of this article, ITCCS $({\mathit{W}}^T,\text{Diag}({\mathit{d}}^t),\frac{1}{N},{\sigma }^2)$ is used to make the description of the output, which is the estimate of $\mathit{g}$, and the parameters in brackets present the input of the algorithm.

4.2. The Fractional Off-Grid Angle Estimator

By fixing the integral virtual channel $\mathit{g}$, and as a result of the equivalence of an elementary row operation and premultiplication by a permutation matrix according to the theory of linear algebra, the Equation (17) can be rewritten as:

$$\begin{array}{cc}\hfill \mathit{y}& =\mathit{W}\mathit{D}\mathit{f}+\mathit{z}\hfill \\ & ={[\mathit{d}(\delta )]}_{\mathcal{R}}\circ {[\mathit{f}]}_{\mathcal{R}}+\mathit{z}\hfill \end{array}$$

(19)

where $\mathcal{R}=\{n_1,n_2,\dots ,n_K\}$ is a set who is constituted by the row indices that are selected from $\mathit{I}$ to $\mathit{W}$ and the operator ∘ denotes the element-wise product. Equation (19) can be solved by the least square (LS) approach.

According to the proposed iteration, the IOTCE algorithm is illustrated in Algorithm 1, which presents a structure of a nested loop. In addition, the corresponding explanation of parameters in the IOTCE algorithm is listed in

Table 1. Explanation of parameters in the iterative off-grid turbo channel estimation algorithm.

Parameters	Explanation
N	the number of antennas at receiver
$\mathit{g}$	the integral virtual channel vector
$\mathit{W}$	the combining matrix at receiver
δ	the fractional off-grid angle
$\mathit{d}$	the corresponding fractional off-grid angle

. The inner loop iterates the integral grid angle between sparsity and linearity, while the integral part and the fractional part are updated in turn during each iteration of the outer loop by following a fixed relationship. As the estimation of integral grid angle improves, the estimation of the fractional off-grid angle enhances via the outer loop.

Algorithm 1 The Iterative Off-grid Turbo Channel Estimation Algorithm

Input: Received signal $\mathit{y}\in {\mathbb{C}}^{RK\times 1}$; Sampling matrix $\mathit{W}\in {\mathbb{C}}^{RK\times N}$, noise power ${\sigma }^2$.

Output: Estimate of $\mathit{g}$, ${\mathit{g}}^t$; Estimation of δ, ${\delta }^t$.

Iteration: Step 1: Initialization $$\begin{array}{c}\hfill t=0,{\delta }^0=0\end{array}$$ Step 2: Estimation of g (1) Fixing δ, $$\begin{array}{cc}\hfill {\mathit{d}}^t& ={\left[1e^{j2\pi {\delta }^t/N}{e}^{j2\pi 2{\delta }^t/N}\cdots e^{j2\pi (N-1){\delta }^t/N}\right]}^T\hfill \\ \hfill {\mathit{g}}^{t+1}& =\text{ITCCS}({\mathit{W}}^T,\text{Diag}({\mathit{d}}^t),\frac{1}{N},{\sigma }^2)\hfill \end{array}$$ Step 3: Estimation of δ (1) Fixing $\mathit{g}$, $$\begin{array}{c}\hfill {[{\mathit{d}}^{t+1}]}_{\mathcal{R}}=\mathit{y}\oslash {[{\mathit{g}}^{t+1}]}_{\mathcal{R}}\end{array}$$ where ⊘ denotes the element-wise division operation. (2) Calculating ${\delta }^{t+1}$ by linear fitting after phase extraction from the equation, $$\begin{array}{c}\hfill {[{\mathit{d}}^{t+1}]}_{\mathcal{R}}={\left[e^{j2\pi (n_1-1){\delta }^{t+1}/N}{e}^{j2\pi (n_2-1){\delta }^{t+1}/N}\cdots e^{j2\pi (n_K-1){\delta }^{t+1}/N}\right]}^T\end{array}$$ Step 4: Iteration of loop IF iteration stopping condition is satisfied output the estimated value. ELSE $t=t+1$. Then, and turn to Step 2.

5. Simulation Results

In this section, we evaluate the performance of the proposed algorithm in the mmWave communication systems. The simulation system consists of a BS and an MS, and the same ULA with $N_B=N_M=N$ half-wavelength spaced antennas and $K_B=K_M=K$ RF chains is configured at both the BS and MS. Typical mmWave channels featuring much stronger LoS paths than NLoS paths are simulated, and a one-path channel model is used to approximate the typical mmWave channel in the low signal-to-noise ratio (SNR) region. The fading weight of the path follows Rayleigh distribution with variance 1, and the continuously-valued AoA/AoD are uniformly-distributed in the range $[-\frac{\pi }{2},\frac{\pi }{2})$.

The simulation results of the average angle estimation error (AAEE), defined as $\mathbb{E}[|\varphi -\widehat{\varphi }|],$ with $\widehat{\varphi }$ being the estimate of the true angle ϕ, are presented in Figure 4, where the on-grid algorithm in [17] and the lower bound of the AAEE for on-grid algorithms, which is $\frac{1}{N}$, are also presented for reference in two cases of $N=1024$ and $N=512$. From the curves in Figure 4, we can observe that the proposed method outperforms the on-grid method and achieves AAEE of 10⁻² at SNR γ = −4 $\mathrm{dB}$. As expected, the AAEE of the proposed algorithm exceeds the lower bound of the on-grid algorithms as SNR increases, which validates the AAEE performance of our algorithm. For example, the gaps between the simulated AAEE and the lower bound are 0.00139 and 0.0007 when γ = 20 $\mathrm{dB}$ for $N=512$ and $N=1024$, respectively. The gap shrinks from 0.00139 to 0.0007 by the increase in N, because the less sparsity ($\lambda =\frac{1}{N}$) can achieve a the better recovery performance at the given samplings according to the theory of CS [20].

The average probability of integral grid point estimation error (APIEE) performance, which is defined as $\mathbb{E}[I({\varphi }_k\ne {\widehat{\varphi }}_k)],$ with $\widehat{{\varphi }_k}$ being the estimate of the real integral grid angle ${\varphi }_k$, is also simulated, and the simulation results are illustrated in Figure 5. As clearly shown, the bigger N leads to the better APIEE performance at a given K. For example, at the APIEE of 10⁻², the algorithm with $N=1024$ almost enhances by 4 $\mathrm{dB}$ with respect to $N=512$.

6. Conclusions

The shortage of spectrum at present will be alleviated by advances in the mmWave communication systems, and directional precoding/beamforming with large antenna arrays appears to be inevitable to support longer outdoor links and to provide sufficient received signal power than before. In this paper, an enhanced CE algorithm is developed for the off-grid problem in the mmWave systems with massive antenna arrays and RF chain constraints. First, we developed an off-grid formulation to catch the quantization problem. Then, an iterative method, which depends on the developed off-grid formulation is proposed to renew and upgrade the CE between the integral and fraction angle information under the Turbo principle. Simulation results show the efficiency and high resolution of the proposed method. For future work, it would be more practical to consider the multi-paths channel with independently but not identically distributed paths, and develop an efficient method to calculate the Log Likelihood Ratio (LLR) information.