An Artificial Intelligence-Assisted Expectation Propagation Detection for MIMO Systems

Abstract

Multiple-input multiple-output (MIMO) technology is one of the key technical approaches to improve the spectrum efficiency of wireless communication. Modern communication systems employ MIMO and high-order quadrature amplitude modulation (QAM) to maximize spectral efficiency. However, with the increase in the number of antennas and modulation orders, it is very challenging to design a low-complexity and high-efficiency MIMO receiver. In recent years, with the rapid development of new technologies such as artificial intelligence, more and more researchers have tried to apply machine learning techniques in the field of communication to break through the performance of traditional communication algorithms. In this paper, we propose a new low-complexity MIMO detection algorithm: an artificial intelligence-assisted expectation propagation (EP) detection algorithm. Neural network models are used to learn and map some of the time-consuming steps in the EP detection algorithm, converting the complex operation process into a few matrix multiplication operations in order to reduce the complexity of the detection algorithm. It is verified that the method proposed in this paper can approximate the performance of the original EP detection algorithm with reduced complexity and is applicable in different scenarios.

1. Introduction

Multiple-input multiple-output (MIMO) technology has been widely applied in wireless communications to improve spectrum efficiency [1]. The main advantage of MIMO systems is their ability to use spatial multiplexing techniques to greatly improve the spectral efficiency of communication systems with bandwidth and power constraints [2]. Currently, modern communication systems often use MIMO and high-order quadrature amplitude modulation (QAM) to improve spectral efficiency.

However, the widespread deployment of high-dimensional MIMO systems is hampered by a number of limitations. In a practical scenario, communication systems are configured with a large number of radio frequency units, and the complexity of the receiver will be very high when performing signal processing [3]. Moreover, the complexity of high-performance maximum likelihood (ML) detection or sphere decoding (SD) increases exponentially with the number of antennas and the modulation order [4]. Therefore, low-complexity and high-performance MIMO detection techniques are of great research importance.

For MIMO communication systems, the ML detector needs to traverse all possible vectors, provided that the receiver obtains channel state information (CSI) and that the transmitted symbols obey a uniform distribution [5]. The ML detector is an NP-hard problem [6] and is too complex to be applied in practice when the number of transmitting antennas is large and high-order modulation is used. Therefore, how to simplify the algorithm of signal detection and approximate the performance of maximum likelihood detection has become a hot issue for the communication community.

The expectation propagation (EP) detection algorithm is a low-complexity algorithm with high performance. The EP algorithm is one of the Bayesian machine learning algorithms. It can efficiently handle continuous distributions through moment matching [7]. Therefore, it has the ability to handle more complex and general approximation functions [8].

The performance of EP detection is very close to ML detection. Even for very large MIMO orders such as 250 transmitting antennas, it only takes 7∼8 iterations to approximate the original distribution, and the complexity of each iteration depends on the inverse of the $n\times n$ matrix [9]. Simulation results show that the number of iterations required by the EP algorithm is independent of the modulation order of the system, and the algorithm complexity is maintained at $O\left(n^3\right)$ [10]. Moreover, EP detection is a soft output algorithm that provides a posterior probability estimate for each received symbol, which can be naturally performed for further channel decoding [11].

The integration of artificial intelligence technology in communication will become a mainstream direction of the sixth-generation mobile communication technology (6G) development [12]. Coupled with the improvement of hardware level and computing power, machine learning technology has been widely used.

Machine learning is a type of artificial intelligence that requires the support of large amounts of data to train models that ideally meet the needs [13]. A neural network is an algorithm in machine learning that belongs to supervised learning. Supervised learning means that the output results of training data are real and certain, and can accurately give the training error and guide the model to train and learn [14]. The neural network is composed of a large number of interconnected nodes. In the process of data transmission iteration, the parameter values of nodes will be gradually optimized and the loss function will be constantly reduced, so as to achieve the purpose of accurately mapping the input data to the output data [15].

Deep learning is a branch of machine learning, which has attracted great attention in recent years, and a large number of experts have conducted in-depth research. Deep learning can be understood as a longitudinal extension of neural network algorithms [16]. Based on the basic neural network, the number of network layers and network reasoning operation process is studied in detail, so that the network model can have more powerful learning ability and the network can learn extremely complex operation processes after a lot of iterative training [17]. In recent years, deep learning has been used in various fields, such as natural language processing, computer vision, and recommendation algorithms.

In the past, most mainstream ideas relied on the powerful learning ability of neural networks to improve the performance of algorithms. The essence of the reasoning process of the network, namely the prediction of new data, is several matrix multiplication operations. Therefore, a novel idea is to learn and train some complex steps in the MIMO detection algorithm through the neural network, converting the complex calculation process into a few matrix multiplication operations, so as to greatly improve the efficiency of the detection algorithm. This strategy not only can effectively reduce the complexity of the detection algorithm in the form of a matrix product, but also can approximate the original detection performance by virtue of the fitting ability of the neural network.

The EP detection algorithm has good performance, but there are still some complicated operations in each iteration, such as matrix inversion, moment matching, etc. These calculations are not conducive to a low-complexity implementation in hardware. Therefore, this paper uses an artificial intelligence approach to optimize some of the complex steps of the EP algorithm to further reduce the complexity of the algorithm and approximate the performance of the EP algorithm. The following are the key contributions of this work:

• This paper analyzes the steps with high complexity, and then uses the neural network model to learn and map some time-consuming steps in the EP detection algorithm, and convert the complex operation process into several matrix multiplication operations to reduce the complexity of the detection algorithm.
• The improved method is tested with different MIMO channel models and verified to have strong robustness as well as adaptability. The performance and applicability of the method are verified in different scenarios.
• Considering the problem of excessive differences in data with different SNRs, SNR is divided into different intervals, and different neural network models are trained on the data with different SNR intervals. The accuracy of the model mapping process is guaranteed.
• A simple structure of 4 × 32 × 16 × 2 is chosen for the neural network model to minimize the complexity of neural network prediction and the space occupied while maintaining sufficient learning capability of the model.

The rest of this paper is organized as follows. In Section 2, we show the MIMO system model and summarize and compare common detection algorithms briefly. In Section 3, we detail the process of the EP detection algorithm. In Section 4, we first analyze the steps with high complexity in the EP algorithm and determine the application strategy of the neural network, then detail the process of model building and training, and finally embed the neural network into the EP detection algorithm. Simulation results and corresponding analysis are shown in Section 5. A conclusion is provided in Section 6.

Notation: Column vectors $\mathbf{x}$ and matrices $\mathbf{X}$ are denoted by bold letters in lower case and in upper case, respectively. $\mathbf{I}$ is an identity matrix. ${\left[·\right]}^{\mathrm{H}}$ and ${\left[·\right]}^{\mathrm{T}}$ are the conjugate transpose and transpose operator, respectively. ${\mathbb{C}}^{m\times n}$ denotes the complex matrix (or vector) with a scale of $m\times n$. $\mathrm{E}\left[·\right]$ and $\mathrm{CoVar}\left[·\right]$ represent the expectation operator and covariance operator, respectively. $\mathcal{C}\mathcal{N}\left(0,{\sigma }^2\right)$ denotes a circularly symmetric complex Gaussian (CSCG) distribution with a mean of 0 and a variance of ${\sigma }^2$.

2. MIMO Symbol Detection

In the MIMO system, an important problem is how to get the signal in the presence of noise, which is the process of MIMO detection. Assuming a MIMO communication system with n transmitting antennas and m receiving antennas, the transmitted symbol vector $\tilde{\mathbf{u}}$ passes through the complex channel matrix $\tilde{\mathbf{H}}$ and the Gaussian white noise $\tilde{\mathsf{\omega }}$ is added to get the output symbol vector $\tilde{\mathbf{y}}$. It can be expressed as:

$$\tilde{\mathbf{y}}=\tilde{\mathbf{H}}\tilde{\mathbf{u}}+\tilde{\mathsf{\omega }},$$

(1)

where $\tilde{\mathbf{u}}={\left[{\tilde{u}}_1,{\tilde{u}}_2,\dots ,{\tilde{u}}_n\right]}^{\mathrm{T}}\in {\mathbb{C}}^{n\times 1}$ is $n\times 1$ i.i.d transmitted symbol vector, where ${\tilde{u}}_i=a_i+j{b}_i\in \tilde{\mathcal{A}}$. $\tilde{\mathbf{H}}\in {\mathbb{C}}^{m\times n}$ is complex MIMO channel matrix. $\tilde{\mathsf{\omega }}={\left[{\tilde{\omega }}_1,{\tilde{\omega }}_2,\dots ,{\tilde{\omega }}_m\right]}^{\mathrm{T}}\in {\mathbb{C}}^{m\times 1}$ is independent additive complex Gaussian white noise with a mean of 0 and variance of ${\sigma }_{\tilde{\omega }}^2$, that is $\mathcal{C}\mathcal{N}\left(0,{\sigma }_{\tilde{\omega }}^2\right)$. $\tilde{\mathbf{y}}={\left[{\tilde{y}}_1,{\tilde{y}}_2,\dots ,{\tilde{y}}_m\right]}^{\mathrm{T}}\in {\mathbb{C}}^{m\times 1}$ is the output symbol vector.

The signal-to-noise ratio (SNR) is an important parameter of communication system [18,19]. A higher SNR means better communication conditions. The SNR of the MIMO communication system can be expressed as:

$$\mathrm{SNR}=10{log}_{10}\frac{n{E}_{\tilde{u}}}{{\sigma }_{\tilde{\omega }}^2},$$

(2)

where $E_{\tilde{u}}$ represents the average energy of transmitted symbols.

The signal detection is that the receiver gets the output symbol $\tilde{\mathbf{y}}$ and restores the transmitted symbol vector $\tilde{\mathbf{u}}$ from $\tilde{\mathbf{y}}$. MIMO detection often adopts Bayesian estimation to estimate the expectation of a posteriori probability. It can be expressed as:

$$\widehat{\mathbf{u}}=\mathrm{E}\left(\tilde{\mathbf{u}}|\tilde{\mathbf{y}}\right)=\underset{{\tilde{u}}_i\in \tilde{\mathbf{u}}}{\int }{\tilde{u}}_{i}p\left({\tilde{u}}_i|\tilde{\mathbf{y}}\right)d{\tilde{u}}_i.$$

(3)

The posterior probability of transmitted symbol vector $\tilde{\mathbf{u}}$ can be expressed as:

$$p\left(\tilde{\mathbf{u}}|\tilde{\mathbf{y}}\right)=\frac{p\left(\tilde{\mathbf{y}}|\tilde{\mathbf{u}}\right)p\left(\tilde{\mathbf{u}}\right)}{p\left(\tilde{\mathbf{y}}\right)}\propto \mathcal{N}\left(\tilde{\mathbf{y}}:\tilde{\mathbf{H}}\tilde{\mathbf{u}},{\sigma }_{\tilde{\omega }}^2\mathrm{I}\right)\prod _{i=1}^n{\mathbb{I}}_{{\tilde{u}}_i\in \tilde{\mathcal{A}}}.$$

(4)

Its a typical variation of Bayesian formula, where $\mathcal{N}\left(\tilde{\mathbf{y}}:\tilde{\mathbf{H}}\tilde{\mathbf{u}},{\sigma }_{\tilde{\omega }}^2\mathbf{I}\right)$ represents a normal distribution with mean $\tilde{\mathbf{H}}\tilde{\mathbf{u}}$ and covariance matrix ${\sigma }_{\tilde{\omega }}^2\mathbf{I}$. $\prod _{i=1}^n{\mathbb{I}}_{{\tilde{u}}_i\in \tilde{\mathcal{A}}}$ means that if ${\tilde{u}}_i\in \tilde{\mathcal{A}}$, then this function value is 1; otherwise, it is 0. ML detection can be expressed as:

$${\widehat{\mathbf{u}}}_{\mathrm{ML}}=\underset{\tilde{\mathbf{u}}\in {\tilde{\mathcal{A}}}^n}{argmax}p\left(\tilde{\mathbf{u}}|\tilde{\mathbf{y}}\right)=\underset{\tilde{\mathbf{u}}\in {\tilde{\mathcal{A}}}^n}{argmin}{∥\tilde{\mathbf{H}}\tilde{\mathbf{u}}-\tilde{\mathbf{y}}∥}^2,$$

(5)

where ${\widehat{\mathbf{u}}}_{\mathrm{ML}}$ is the result of ML detection, $\tilde{\mathbf{u}}$ is the complex transmitted symbol vector taken from an n-dimensional complex symbol set ${\tilde{\mathcal{A}}}^n$, $\tilde{\mathbf{H}}$ is the $m\times n$ MIMO complex channel matrix, and $\tilde{\mathbf{y}}$ is the channel observation vector.

Equation (1) is a complex system model. It can be converted to an equivalent real system model [20]. After transforming the matrix and signal by

$$\mathbf{y}={\left[\begin{array}{cc}\mathcal{R}{\left(\tilde{\mathbf{y}}\right)}^{\mathrm{T}}& \mathcal{I}{\left(\tilde{\mathbf{y}}\right)}^{\mathrm{T}}\end{array}\right]}^{\mathrm{T}},$$

(6)

$$\mathbf{u}={\left[\begin{array}{cc}\mathcal{R}{\left(\tilde{\mathbf{u}}\right)}^{\mathrm{T}}& \mathcal{I}{\left(\tilde{\mathbf{u}}\right)}^{\mathrm{T}}\end{array}\right]}^{\mathrm{T}},$$

(7)

$$\mathsf{\omega }={\left[\begin{array}{cc}\mathcal{R}{\left(\mathsf{\omega }\right)}^{\mathrm{T}}& \mathcal{I}{\left(\mathsf{\omega }\right)}^{\mathrm{T}}\end{array}\right]}^{\mathrm{T}},$$

(8)

$$\mathbf{H}=\left[\begin{array}{cc}\mathcal{R}\left(\tilde{\mathbf{H}}\right)& -\mathcal{I}\left(\tilde{\mathbf{H}}\right)\\ \mathcal{I}\left(\tilde{\mathbf{H}}\right)& \mathcal{R}\left(\tilde{\mathbf{H}}\right)\end{array}\right],$$

(9)

the equivalent real system model can be expressed as:

$$\mathbf{y}=\mathbf{Hu}+\mathsf{\omega }.$$

(10)

In the real system model, the noise power is ${\sigma }_{\omega }^2=0.5{\sigma }_{\tilde{\omega }}^2$, the average energy of transmitted symbols is $E_u=0.5E_{\tilde{u}}$. In the rest of this paper, we adopt the real-valued channel model formulation in (10). We will use the EP detection algorithm to obtain the detected transmitted signal $\widehat{\mathbf{u}}$ from the received signal $\mathbf{y}$. Then we use the neural network to optimize the EP detection algorithm and compare the performance of both in this real-valued channel model.

The detection algorithms commonly used in communication systems are the following categories: linear detection, nonlinear detection, optimal detection algorithm, and so on [21]. The summary of common MIMO detection algorithms is shown in

Table 1. Summary of common MIMO detection algorithms.

Algorithms	Descriptions	Complexity
Maximum Likelihood (ML)	Best performance, highest complexity	$O\left(2^{n\times \left|\mathcal{A}\right|}\right)$
Maximum a Posteriori (MAP)	Regularized maximum likelihood	$O\left(2^{n\times \left|\mathcal{A}\right|}\right)$
Matched Filter (MF)	Linear detectors (LD)	$O\left(n\times m\right)$
Zero Forcing (ZF)		$O\left(n^3\right)$
Minimum Mean Square Error (MMSE)		$O\left(n^3\right)$
Ordered Successive Interference Cancellation (OSIC)	Interference elimination	$O\left(n^4\right)$
Sphere Decoding (SD)	Tree based search	Unstable, depends on simulation
Gaussian Tree Approximation (GTA)		$O\left(n^3\right)$
Belief Propagation (BP)	Probability estimation theory	$O\left(n^2{\left|\mathcal{A}\right|}^2\right)$
Probabilistic Data Association (PDA)		$O\left(n^3\right)$
Construct optimization problem to solve	Convex optimization theory	Unstable, depends on simulation

.

3. Expectation Propagation Detection Algorithm

The EP detection algorithm uses Gaussian approximation $q_{\mathrm{EP}}\left(\mathbf{u}\right)=\mathcal{N}\left(\mathbf{u}:{\mathsf{\mu }}_{\mathrm{EP}},{\Sigma }_{\mathrm{EP}}\right)$ to approximate the signal posterior distribution $p\left(\mathbf{u}|\mathbf{y}\right)$[22]. Therefore, the optimal EP solution is that the mean ${\mathsf{\mu }}_{\mathrm{EP}}$ and variance ${\Sigma }_{\mathrm{EP}}$ of Gaussian approximation $q_{\mathrm{EP}}\left(\mathbf{u}\right)$ agree with those of the posteriori distribution $p\left(\mathbf{u}|\mathbf{y}\right)$. The mean and variance can be expressed as:

$${\mathsf{\mu }}_{\mathrm{EP}}={\mathrm{E}}_{p\left(\mathbf{u}|\mathbf{y}\right)}\left[\mathbf{u}\right],$$

(11)

$${\Sigma }_{\mathrm{EP}}=\mathrm{CoVa}{\mathrm{r}}_{p\left(\mathbf{u}|\mathbf{y}\right)}\left[\mathbf{u}\right].$$

(12)

It takes ${\left|\mathcal{A}\right|}^n$ operations to directly calculate the moments of the signal’s posteriori distribution $p\left(\mathbf{u}|\mathbf{y}\right)$. $\mathcal{A}$ represents a constellation symbol set. n is the number of transmitting antennas. EP iteration is a polynomial complexity with n to appreciate (11) and (12). When the iteration is over, the EP detection will get the hard decision output. It can be expressed as:

$${\widehat{u}}_{k,\mathrm{EP}}=arg\underset{u_k\in \mathcal{A}}{min}{\left|u_k-u_{k,\mathrm{EP}}\right|}^2,k=1,2,\dots ,2n.$$

(13)

The procedure of the EP detection algorithm is as follows:

First, each non-Gaussian factor is replaced with a non-normalized Gaussian factor $q\left(\mathbf{u}\right)$, $q\left(\mathbf{u}\right)$ can be expressed as:

$$q\left(\mathbf{u}\right)\propto \mathcal{N}\left(y:\mathbf{Hu},{\sigma }_{\omega }^2\mathbf{I}\right)\prod _{k=1}^{2n}{e}^{{\gamma }_k{u}_k-\frac{1}{2}{\lambda }_k{u}_k^2},$$

(14)

where ${\gamma }_k$ and ${\lambda }_k$ are real constants greater than 0. $q\left(\mathbf{u}\right)$ is a Gaussian distribution with the mean vector $\mu$ and the covariance matrix $\Sigma$. $\mu$ and $\Sigma$ can be expressed as:

$$\Sigma ={\left({\sigma }_{\omega }^{-2}{\mathbf{H}}^{\mathrm{T}}\mathbf{H}+\mathrm{diag}\left(\lambda \right)\right)}^{-1},$$

(15)

$$\mu =\Sigma \left({\sigma }_{\omega }^{-2}{\mathbf{H}}^{\mathrm{T}}\mathbf{y}+\gamma \right),$$

(16)

where $\lambda ={\left[{\lambda }_1,{\lambda }_2,\dots ,{\lambda }_N\right]}^{\mathrm{T}}$, $\gamma ={\left[{\gamma }_1,{\gamma }_2,\dots ,{\gamma }_N\right]}^{\mathrm{T}}$. ${\left(·\right)}^{\mathrm{T}}$ represents the conjugate transpose operation. $\mathrm{diag}\left(·\right)$ represents that if the input is a square matrix, take its diagonal elements as a vector, and if the input is a vector, it is expanded as diagonal elements into a square matrix.

The EP detection updates $\left({\gamma }_k,{\lambda }_k\right),k=1,2,\dots ,2n$ iteratively, and the mean ${\mathsf{\mu }}_{\mathrm{EP}}$ and variance ${\Sigma }_{\mathrm{EP}}$ of the non-normalized Gaussian factor $q\left(\mathbf{u}\right)$ will gradually approximate the mean and variance of the original signified posterior probability distribution $p\left(\mathbf{u}|\mathbf{y}\right)$ with polynomial complexity, that is, to get the optimal EP solution [23]. All parameters related to the index k of the sent symbol are first initialized before iterations. Initialize ${\gamma }_k=0$ and ${\lambda }_k=E_s^{-1}$. $E_u$ is the symbol average energy (J/symbol). During each EP iteration, $\left({\gamma }_k^{\left(t+1\right)},{\lambda }_k^{\left(t+1\right)}\right),k=1,2,\dots ,2n$ will be updated in parallel, where $t\in \left[1,iter\right]$ represents the number of rounds of the current EP iteration and $iter$ is the maximum number of internal iterations of EP. Given the marginal distribution of the distribution $q^{\left(t\right)}\left(\mathbf{u}\right)$, namely $q_k^{\left(t\right)}\left(u_k\right)=\mathcal{N}\left(u_k:{\mu }_k^{\left(t\right)},{\sigma }_k^{2\left(t\right)}\right)$, the pair $\left({\gamma }_k^{\left(t+1\right)},{\lambda }_k^{\left(t+1\right)}\right)$ will be updated iteratively as follows [24]:

(1)

Obtain the cavity marginal distribution $q^{\left(t\right)\backslash k}\left(u_k\right)$ based on $q^{\left(t\right)}\left(u_k\right)$, namely approximate logarithmic likelihood function:

$$q^{\left(t\right)\backslash k}\left(u_k\right)=\frac{q^{\left(t\right)}\left(u_k\right)}{exp\left({\gamma }_k^{\left(t\right)}{u}_k-\frac{1}{2}{\lambda }_k^{\left(t\right)}{u}_k^2\right)}\sim \mathcal{N}\left(u_k:{\epsilon }_k^{\left(t\right)},{\upsilon }_k^{2\left(t\right)}\right).$$

(17)

It follows a Gaussian distribution with mean ${\epsilon }_k^{\left(t\right)}$ and variance ${\upsilon }_k^{2\left(t\right)}$.

(2)

Compute the mean ${\epsilon }_k^{\left(t\right)}$ and variance ${\upsilon }_k^{2\left(t\right)}$ of the cavity marginal distribution $q^{\left(t\right)\backslash k}\left(u_k\right)$:

$${\upsilon }_k^{2\left(t\right)}=\frac{{\sigma }_k^{2\left(t\right)}}{\left(1-{\sigma }_k^{2\left(t\right)}{\lambda }_k^{\left(t\right)}\right)},$$

(18)

$${\epsilon }_k^{\left(t\right)}={\upsilon }_k^{2\left(t\right)}\left(\frac{u_k^{\left(t\right)}}{{\sigma }_k^{2\left(t\right)}}-{\gamma }_k^{\left(t\right)}\right).$$

(19)

(3)

Compute the mean ${\mu }_{pk}^{\left(t\right)}$ and variance ${\sigma }_{pk}^{2\left(t\right)}$ of the approximate posterior probability distribution ${\widehat{p}}^{\left(t\right)}\left(u_k\right)$. The distribution ${\widehat{p}}^{\left(t\right)}\left(u_k\right)$ can be expressed as:

$${\widehat{p}}^{\left(t\right)}\left(u_k\right)\propto q^{\left(t\right)\setminus k}\left(u_k\right){\mathbb{I}}_{u_k\in \mathcal{A}}.$$

(20)

In order to ensure the stability of the algorithm, we set a constraint on ${\sigma }_{pk}^{2\left(t\right)}$, namely ${\sigma }_{pk}^{2\left(t\right)}=max\left(\delta ,\mathrm{Va}{\mathrm{r}}_{{\widehat{p}}^{\left(t\right)}}\left(u_k\right)\right)$ according to [10], where $\delta =5\times {10}^{-7}$.

(4)

Update the pair $\left({\gamma }_k^{\left(t+1\right)},{\lambda }_k^{\left(t+1\right)}\right)$:

$${\gamma }_k^{\left(t+1\right)}=\frac{{\mu }_{pk}^{\left(t\right)}}{{\sigma }_{pk}^{2\left(t\right)}}-\frac{{\epsilon }_k^{\left(t\right)}}{{\upsilon }_k^{2\left(t\right)}},$$

(21)

$${\lambda }_k^{\left(t+1\right)}=\frac{1}{{\sigma }_{pk}^{2\left(t\right)}}-\frac{1}{{\upsilon }_k^{2\left(t\right)}}.$$

(22)

According to the moment matching principle, the second moment of the cavity marginal distribution ${\widehat{q}}^{\left(t\right)\backslash k}\left(u_k\right)$ is equal to the second moment of the approximate posterior probability distribution ${\widehat{p}}^{\left(t\right)}\left(u_k\right)$. That is, make the unnormalized Gaussian distribution ${\widehat{q}}^{\left(t\right)\backslash k}\left(u_k\right)$ as shown in (23) have mean ${\mu }_{pk}^{\left(t\right)}$ and variance ${\sigma }_{pk}^{2\left(t\right)}$.

$${\widehat{q}}^{\left(t\right)\backslash k}\left(u_k\right)=q^{\left(t\right)\backslash k}\left(u_k\right)exp\left({\gamma }_k^{\left(t+1\right)}{u}_k-\frac{1}{2}{\lambda }_k^{\left(t+1\right)}{u}_k^2\right).$$

(23)

During the update process, it may occur that ${\lambda }_k^{\left(t+1\right)}$ is negative, however, this parameter is an accuracy term and should be positive. The reason for this situation is that there is no pair of $\left({\gamma }_k^{\left(t+1\right)},{\lambda }_k^{\left(t+1\right)}\right)$ can make the variance of $q\left(\mathbf{u}\right)$ approximate to the variance of the original distribution, that is, the moment matching condition cannot be satisfied. In this case, $\gamma$ and $\lambda$ will not be updated. Just keep the parameter values the same as before the iteration. That is ${\gamma }_k^{\left(t+1\right)}={\gamma }_k^{\left(t\right)}$, ${\lambda }_k^{\left(t+1\right)}={\lambda }_k^{\left(t\right)}$.

(5)

In order to improve the robustness of the algorithm, $\left({\gamma }_k^{\left(t+1\right)},{\lambda }_k^{\left(t+1\right)}\right)$ is smoothed. It can be expressed as:

$${\gamma }_k^{\left(t+1\right)}=\beta \left(\frac{{\mu }_{pk}^{\left(t\right)}}{{\sigma }_{pk}^{2\left(t\right)}}-\frac{{\epsilon }_k^{\left(t\right)}}{{\upsilon }_k^{2\left(t\right)}}\right)+\left(1-\beta \right){\gamma }_k^{\left(t\right)},$$

(24)

$${\lambda }_k^{\left(t+1\right)}=\beta \left(\frac{1}{{\sigma }_{pk}^{2\left(t\right)}}-\frac{1}{{\upsilon }_k^{2\left(t\right)}}\right)+\left(1-\beta \right){\lambda }_k^{\left(t\right)},$$

(25)

where $\beta \in \left[0,1\right]$ is the smoothness coefficient.

When the change of the mean and covariance of the posterior probability distribution between two iterations is less than ${10}^{-4}$ or the maximum number of iterations has been reached, the iteration is stopped.

4. Artificial Intelligence-Assisted Expectation Propagation Detection Algorithm

4.1. Application Strategy of Neural Network

In the EP detection algorithm, the most complex and time-consuming step is to calculate the mean vector $\mu$ and covariance matrix $\Sigma$ of Gaussian approximation $q\left(\mathbf{u}\right)$. In this process, there is an inversion operation, which will consume a lot of computation resources, as shown in (15) and (16). Secondly, in the process of EP inner loop iteration, it is relatively computation-consuming to calculate the mean ${\epsilon }_k^{\left(t\right)}$ and variance ${\upsilon }_k^{2\left(t\right)}$ of the cavity marginal distribution $q^{\left(t\right)\backslash k}\left(u_k\right)$ and update the pair $\left({\gamma }_k^{\left(t+1\right)},{\lambda }_k^{\left(t+1\right)}\right)$, as shown in (18), (19), (21), and (22).

Therefore, we try to simplify the above three steps by using a neural network. We take four variables, namely the mean ${\mu }_{pk}^{\left(t\right)}$ and variance ${\sigma }_{pk}^{2\left(t\right)}$ of ${\widehat{p}}^{\left(t\right)}\left(u_k\right)$, and the mean ${\epsilon }_k^{\left(t\right)}$ and variance ${\upsilon }_k^{2\left(t\right)}$ of the cavity marginal distribution $q^{\left(t\right)\backslash k}\left(u_k\right)$, as the input of the neural network, and the mean ${\epsilon }_k^{\left(t+1\right)}$ and variance ${\upsilon }_k^{2\left(t+1\right)}$ of the cavity marginal distribution $q^{\left(t+1\right)\backslash k}\left(u_k\right)$ after the iteration as the output of the neural network. The relatively complicated steps in the EP detection process can be skipped, and the probability density function of the normal distribution can be generated directly according to the mean ${\epsilon }_k^{\left(t+1\right)}$ and variance ${\upsilon }_k^{2\left(t+1\right)}$ of the output results of the neural network, so as to obtain the detection result.

The essence of a neural network is that the input data are calculated through network nodes to get the output and the intermediate calculation process can be equivalent to a matrix multiplication operation. So, the iteration process of EP detection can actually be equivalent to several matrix multiplication operations with the aid of neural networks, and the operation complexity can be greatly reduced compared with the complex formula calculation directly. Another advantage of neural networks is that no matter how big the data size is, the calculation process is still several steps of matrix operation, which can be effectively calculated in parallel. In the case of large amounts of data, the operation efficiency can be greatly improved.

4.2. Training Data Collection and Model Structure Design

The first step is to collect the required training data and design the model structure on the basis of comprehensive consideration of computational complexity and learning ability.

In the data collection stage, six arrays are initialized before the simulation to store the six variable data to be collected. In the simulation process, the data to be mapped during each EP iteration are saved. However, in the scenario with different SNRs, the magnitude of the data to be mapped by the neural network is quite different. For example, when SNR is 10, the order of magnitude of the variance ${\upsilon }_k^{2\left(t+1\right)}$ of cavity marginal distribution $q^{\left(t+1\right)\backslash k}\left(u_k\right)$, which neural network needs to output, is roughly ${10}^{-3}$. Correspondingly, when the SNR is 30, the order of magnitude of ${\upsilon }_k^{2\left(t+1\right)}$ is roughly ${10}^{-5}$. In order to obtain better training results, SNR is divided into different intervals, and different neural network models are trained on the data with different SNR intervals. We train a total of four models and divide the SNR as follows: we train the first model on the data with $\mathrm{SNR}<15\mathrm{dB}$, train the second model on the data with $15\mathrm{dB}⩽\mathrm{SNR}<19\mathrm{dB}$, train the third model on the data with $19\mathrm{dB}⩽\mathrm{SNR}<23\mathrm{dB}$, and train the last model on the data with $\mathrm{SNR}⩾23\mathrm{dB}$.

Finally, there are 307,200 pieces of data in each SNR interval, and each piece of data has 6 features, that is, the data scale is 307,200 × 6. The first four features are inputs and the last two features are outputs.

The essence of forward propagation of the neural network model is matrix multiplication, and the parameters of the matrix are the weights that the model needs to be trained. The number of hidden layers determines the times of matrix multiplication required in the prediction, and to a large extent determines the algorithm complexity of the model prediction once. In addition, the number of nodes in each layer of the neural network will affect the learning ability of the model. The number of nodes of the model is more, the learning and fitting ability will be stronger, and the amount of data required will be larger; otherwise, the phenomenon of overfitting will occur. If the number of nodes is too small, the learning ability of the model is limited, it is not easy to learn the complex distribution, and it is prone to the underfitting phenomenon.

After many attempts and comprehensive consideration of computational complexity and learning ability, the final structure of the model has four nodes in the input layer, corresponding to four input variables ${\sigma }_{pk}^{2\left(t\right)}$, ${\mu }_{pk}^{\left(t\right)}$, ${\upsilon }_k^{2\left(t\right)}$ and ${\epsilon }_k^{\left(t\right)}$, two hidden layers of thirty-two and sixteen nodes, and two nodes in the output layer, corresponding to the two output variables ${\upsilon }_k^{2\left(t+1\right)}$ and ${\epsilon }_k^{\left(t+1\right)}$.

4.3. Model Training and Application

The neural network model designed above is adopted to learn and map the steps required for optimization, and the trained model parameters are obtained. The process is shown in Figure 1.

First, the collected training data are imported and divided into input, output, training, and test data. The divided test set accounts for 20% of the total dataset to ensure the generalization ability of the model.

Then, we adopt the z-score method for data standardization to alleviate the impact of data dimension. Z-score standardization will normalize the data into a distribution with a mean of 0 and a standard deviation of 1. It can be expressed as:

$$x_{\mathrm{std}}=\frac{x-{\mu }_{\mathrm{train}}}{{\sigma }_{\mathrm{train}}},$$

(26)

where ${\mu }_{\mathrm{train}}$ and ${\sigma }_{\mathrm{train}}$ are the mean and standard deviation of the input data of the training set, respectively. We use them to standardize the input data of the training and test set. $x_{\mathrm{std}}$ is the standardized result.

Then the data format is processed. The Pytorch implementation of the neural network requires the data in tensor format. Data with the tensor format can be computed faster in a neural network and can create higher-dimensional matrices and vectors. Therefore, the dataset needs to be converted to the tensor format. In addition, we convert the data format to float32. The default double data format cannot be trained in the Pytorch framework. Next, a data loader is created to train the data by passing it in batches in an iterative manner.

Finally, the neural network models are built. We adopt multilayer perceptron (MLP) with three fully connected layers, 4 × 32, 32 × 16, and 16 × 2, to realize the learning of data and ensure the simplicity of reasoning. We use ReLU() as the activation function to maximize the speed of computation and training convergence. It can be expressed as:

$$\mathrm{ReLU}\left(x\right)=\left\{\begin{array}{c}\hfill x,ifx>0\hfill \\ \hfill 0,ifx⩽0\hfill \end{array}\right..$$

(27)

We use Adam optimizer and the initial learning rate is set to 0.001. The L1 loss function, namely the minimum absolute deviation, is adopted as the error loss function. It represents the average of the sum of the absolute values of the difference between the true value and the predicted value. The model trained with this loss function is more robust to some extent. It can be expressed as:

$$\mathrm{MAE}=\frac{{\displaystyle \sum _{i=1}^n}\left|y_i-{\widehat{y}}_i\right|}{n}.$$

(28)

Environment parameters and neural network training parameters are shown in

Table 2. Parameters of neural network module.

Parameters	Values
Programming language	Python3.7
Implementation framework	Pytorch
CUDA version	10.1
Training equipment	GPU: RTX 2080 Ti
Epochs of training	200
Optimizer	Adam
Initial learning rate	0.001
Activation function	ReLU
Loss function	L1 norm

.

After the trained model is obtained, it is necessary to further transform the complex calculation process of the EP detection algorithm into the forward propagation process of the neural network. According to SNR, the corresponding model parameters and the corresponding mean and standard deviation for standardization are imported. In the EP detection iteration process, when obtaining the mean ${\mu }_{pk}^{\left(t\right)}$ and variance ${\sigma }_{pk}^{2\left(t\right)}$ of the approximate posterior probability distribution ${\widehat{p}}^{\left(t\right)}\left(u_k\right)$ and the mean ${\epsilon }_k^{\left(t\right)}$ and variance ${\upsilon }_k^{2\left(t\right)}$ of the last round cavity marginal distribution $q^{\left(t\right)\backslash k}\left(u_k\right)$, the format of these four variables is processed into the neural network input format, and then the standardization is carried out. After that, the matrix multiplication of input and the weight parameters of the neural network is performed, that is, the prediction process of the neural network is reproduced. The process of neural network prediction is as follows:

$$\mathrm{ReLU}\left(\mathrm{Inpu}{\mathrm{t}}^{2n\times 4}·{\mathbf{W}}_1^{4\times 32}+{\mathbf{b}}_1^{2n\times 32}\right)={\mathrm{Hidden}}_1^{2n\times 32},$$

(29)

$$\mathrm{ReLU}\left({\mathrm{Hidden}}_1^{2n\times 32}·{\mathbf{W}}_2^{32\times 16}+{\mathbf{b}}_2^{2n\times 16}\right)={\mathrm{Hidden}}_2^{2n\times 16},$$

(30)

$$\mathrm{ReLU}\left({\mathrm{Hidden}}_2^{2n\times 16}·{\mathbf{W}}_3^{16\times 2}+{\mathbf{b}}_3^{2n\times 2}\right)={\mathrm{Predict}}_2^{2n\times 2},$$

(31)

where n is the num of transmitting antennas. Finally, the output results are the mean ${\epsilon }_k^{\left(t+1\right)}$ and variance ${\upsilon }_k^{2\left(t+1\right)}$ of the cavity marginal distribution $q^{\left(t+1\right)\backslash k}\left(u_k\right)$ after one iteration.

5. Simulation Results

In this section, we compare and analyze the proposed detection algorithm and the original EP detection algorithm in a Rayleigh fading MIMO channel.

In the scenario, there are eight transmitting antennas and eight receiving antennas, and the Rayleigh fading MIMO channel with a mean of 0 and variance of 1 is adopted. Cyclic redundancy check addition, code block segmentation, LDPC coding, and rate matching are performed on the transmitted signals in turn. The modulation modes are 64-QAM and 256-QAM for comparison. At the receiving end, rate matching, LDPC, block segmentation, and cyclic redundancy check are decoded in order. We use the original EP detection algorithm and neural network-assisted EP detection algorithm, respectively, to test and compare the simulation results of the bit error rate (BER) of the soft decision.

Figure 2 and Figure 3 show the comparison of BER detection results after four iterations in the Rayleigh fading MIMO channel scenario with eight transmitting antennas and eight receiving antennas. 64-QAM and 256-QAM are adopted, respectively. “EP” represents the original EP detection. “MLP” represents the EP detection algorithm assisted by the neural network that we propose. When SNR is less than 20 dB, the EP detection algorithm assisted by the neural network can approximate the performance of the original EP algorithm. The results of the fourth iteration of the two algorithms are very close, indicating that the neural network can learn the core steps of the EP algorithm ideally. However, in the case of high SNR ($\mathrm{SNR}⩾20\mathrm{dB}$), the magnitude of the data that the neural network needs to learn is too little, as little as ${10}^{-5}$ order of magnitude size. It requires higher precision to fit the mapping data. In addition, the neural network needs to take into account the generalization ability of the model, and the characteristics of the model lead to its own errors. With the improvement of mapping data accuracy requirements, the tolerance of errors existing in neural networks will also decrease. Therefore, the error of the final detection performance is slightly larger, but the error is also within an acceptable range, and the effect of two iterations of the original EP algorithm can still be achieved. The results show that the EP algorithm assisted by neural networks is a feasible strategy.

Since the simulation data of the eight-input, eight-output Rayleigh fading MIMO channel scenario with 64-QAM constellation is used as the training data, in order to verify the applicability of the neural network-assisted EP detection algorithm in more different scenarios, we use a correlated Rayleigh fading MIMO channel scenario of four inputs and eight outputs, which is completely different from the training data, for testing. The correlation coefficient of adjacent antennas is 0.5. The 64-QAM and 256-QAM modulation modes are adopted, respectively. The simulation results are shown in Figure 4 and Figure 5.

When SNR is less than 20 dB, the method proposed in this paper can still approximate the original EP algorithm, and the results of the fourth iteration of the two algorithms are very close, indicating that the neural network has a good training effect on the core process of EP detection, and is applicable to different scenarios, with strong generalization ability. In the case of high SNR ($\mathrm{SNR}⩾20\mathrm{dB}$), from the perspective of the iterative process, it is found that the performance of the fourth iteration is worse than that of the third iteration once in a while. The reason is that the third iteration has reached a good performance, and the next iteration requires more rigorous calculations to further improve the performance. However, the matrix multiplied by each iteration of the proposed algorithm is the same content, which lacks the effect of fine-tuning. In addition, in the scenario of high SNR, it is difficult for the accuracy of the neural network to meet the requirements, so further iteration may lead to performance degradation. From the perspective of performance, due to the model characteristics of the neural network itself and the improvement of data accuracy requirements, the performance error is slightly larger than that of the original EP algorithm, but still maintains a good detection effect, which fully verifies the applicability of the model for different scenarios.

Next, the complexity is analyzed and compared. For one iteration, the original EP algorithm firstly should calculate (21) and (22) to update $\left({\gamma }_k^{\left(t+1\right)},{\lambda }_k^{\left(t+1\right)}\right)$. Then, we calculate (24) and (25) to smooth $\left({\gamma }_k^{\left(t+1\right)},{\lambda }_k^{\left(t+1\right)}\right)$ and deal with the case where ${\lambda }_k^{\left(t+1\right)}$ is negative. After that, (15) and (16) are calculated to find the mean vector $\mu$ and covariance matrix $\Sigma$ of the non-normalized Gaussian approximation $q\left(\mathbf{u}\right)$. This process involves matrix inversion and matrix multiplication, the complexity is $O\left(n^3\right)+O\left(n\right)$. Finally, according to (18) and (19), the mean ${\epsilon }_k^{\left(t\right)}$ and variance ${\upsilon }_k^{2\left(t\right)}$ of the cavity marginal distribution $q^{\left(t\right)\backslash k}\left(u_k\right)$ are obtained. Therefore, the computational complexity of one iteration of the original EP algorithm is $O\left(n^3\right)$.

The iterative process of the neural network-assisted EP algorithm only needs to go through three matrix multiplication, matrix addition, and maximizing operations. According to (29)–(31), the main computational complexity of this process can be expressed as $O\left(2n\times 4\times 32\right)+O\left(2n\times 32\times 16\right)+O\left(2n\times 4\times 2\right)$. After simplification, the final computational complexity is $O\left(n\right)$. The complexity is greatly reduced compared to MAP and the original EP algorithm. The computational complexity comparison of the three methods is shown in

Table 3. Computational complexity comparison.

Algorithms	Complexity
Maximum a Posteriori (MAP)	$O\left(2^{n\times \left|\mathcal{A}\right|}\right)$
The original EP detection	$O\left(n^3\right)$
The neural network assisted EP detection	$O\left(n\right)$

.

However, the reduction in computational complexity inevitably takes up additional space. The main additional space occupied in our proposed algorithm comes from the parameters of the neural network models. The space required by a neural network model is three weight matrices $\mathbf{W}$ with dimensions $4\times 32$, $32\times 16$, and $16\times 2$, and three bias vectors $\mathbf{b}$ with dimension $1\times 32$, $1\times 16$, and $1\times 2$.

6. Conclusions

In this paper, an EP detection algorithm with outstanding performance and low complexity is optimized. We analyze the steps with high complexity, then use the neural network model to learn and map some time-consuming steps in the EP detection algorithm, and convert the complex operation process into several matrix multiplication operations to reduce the complexity of the detection algorithm. Finally, the performance and applicability of the method are verified in different scenarios. The simulation results show that the neural network-assisted EP detection algorithm can approximate the performance of the original EP algorithm when the SNR is less than 20 dB and can learn the core steps of the EP algorithm more ideally. However, in the case of high SNR ($\mathrm{SNR}⩾20\mathrm{dB}$), the magnitude of the data that the neural network needs to learn is too small, as small as ${10}^{-5}$ orders of magnitude. It requires higher precision to fit the mapping data. In addition, the neural network needs to take into account the generalization ability of the model, and the characteristics of the model lead to its own errors. With the improvement of mapping data accuracy requirements, the tolerance of errors existing in neural networks will also decrease. Therefore, the error of the final detection performance is slightly larger, but the error is also within an acceptable range, and the detection performance still maintains a good level. The results show that EP algorithm assisted by a neural network is a feasible strategy. It can guarantee that the performance is close to the original EP detection algorithm, and the computational complexity can be reduced from the original $O\left(n^3\right)$ to $O\left(n\right)$.

However, performance can still be improved in scenarios with high SNRs. In the future, we can try to adopt other models to achieve a better mapping effect or use different models for training separately in each iteration so as to mitigate performance loss in scenarios with high SNRs.