1. Introduction
In recent years, wireless communication networks in general, and mobile networks in particular, have undergone rapid development, becoming an essential part of our daily lives. They enable easy and convenient connectivity and information exchange. To meet the growing demands of users, researchers have been continuously exploring innovative methods to address the current challenges and improve the performance of these networks [
1,
2]. Currently, several approaches are being pursued to tackle these issues. The first approach is utilizing various channel coding techniques such as Turbo, Polar, and LDPC codes [
3,
4]. The use of these techniques can improve the reliability of data transmission. Secondly, efficient spectrum utilization can be provided by using different types of multicarrier modulation schemes such as orthogonal frequency division multiplexing (OFDM), filter-bank multicarrier (FBMC), generalized frequency sivision multiplexing (GFDM), and spectrally efficient frequency division multiplexing (SEFDM) [
1,
2]. However, these two approaches make the design of the transmitter and receiver much more complicated. Therefore, a promising and emerging direction of applying machine learning techniques into wireless communication systems becomes more and more popular. This direction introduces a new paradigm that holds great promise, not only due to its simplicity in design but also due to its expected capability to break through common limitations in communication systems, such as the Shannon limit. In conjunction with advancements in the field of deep learning, numerous studies have proposed the use of deep learning for end-to-end optimization of communication systems [
5,
6,
7,
8]. In addition, there are several advanced technologies recently proposed for use in 5G wireless networks, such as hybrid beamforming [
9], rate splitting multiple access (RSMA) [
10,
11], and non-orthogonal multiple access (NOMA) [
12]. These advanced techniques have the potential to revolutionize the operation of 5G networks, providing higher data rates, improved connectivity, and enhanced user experiences, making 5G technology even more powerful and versatile.
Among these approaches, autoencoders have gained significant attention [
5,
6]. In contrast to conventional communication systems, autoencoders enable global optimization of both the transmitter and receiver for any channel model, without being constrained by separately optimizing individual components such as channel coding, modulation, and channel equalization. The fundamental idea behind autoencoders is to learn compact representations of data by training a neural network to reconstruct its input at the output layer. In the context of communication systems, this concept is extended to optimize the entire transmission process, from the encoding of information at the transmitter to the decoding at the receiver, in an integrated and adaptive manner. By treating the entire communication system as a single neural network, autoencoders have the potential to overcome the limitations imposed by traditional modular approaches, where each component is optimized independently, often resulting in suboptimal performance. One of the significant advantages of using autoencoders in communication systems is their ability to capture complex dependencies and adapt to different channel conditions. Conventional communication systems often rely on handcrafted algorithms and mathematical models to deal with various impairments in the transmission channel. However, these models may not fully capture the intricate dynamics of real-world channels, leading to suboptimal performance. Autoencoders, on the other hand, have the inherent capability to learn and adapt to channel characteristics through training on large amounts of data. This adaptability enables them to effectively deal with channel variations and imperfections, resulting in improved performance and robustness.
Despite the great advantages of autoencoders, the number of studies evaluating the performance of autoencoders for specific wireless systems is still limited. Some of them are highlighted as follows. In [
6], the authors presented the fundamental concepts of autoencoders and apply them to single-carrier modulation systems such as QPSK, and BPSK, in multipath channels. In [
5], a combined scheme between autoencoders and OFDM modulation was proposed. This combination had improved the BLER performance by 1–2 dB depending on the constellation used. In [
13], an autoencoder using convolutional neural networks (CNN) and the simultaneous perturbation stochastic approximation algorithm (SPSA) in place of backpropogation was proposed. This combination can potentially enhance the performance of the autoencoder by leveraging the strengths of both CNNs and SPSA. In addition, [
14] proposed low-complexity autoencoder-based end-to-end learning of coded communications systems. The use of the proposed autoencoder helps reduce complexity in the design of transmitters and receivers without compromising system performance.
For high-speed wireless communication standards such as 4G LTE, 5G NR, and WiFi, the LDPC coding is chosen. However, the LDPC coding block and the modulation block are separately optimized. There is an idea of replacing the combination of LDPC coding and modulation scheme by an single autoencoder. It is unclear whether these autoencoders can take place of the state-of-the-art LDPC coding under the conditions of additive white Gaussian noise and various fading channels (especially in low-signal-to-noise-ratio scenarios). This study aims to answer the question by conducting the performance evaluation of autoencoders and LDPC coding in single-carrier and OFDM-based wireless communication systems. The main contributions of this paper are summarized as follows. For single-carrier systems, we present the performance evaluation of autoencoders compared with LDPC coding in terms of BLER metrics in the conditions of AWGN. For OFDM-based systems, the performance comparison of OFDM-based autoencoders is conducted in 5G NR fading channels (TDLA30-10). The experiments are conducted for various combinations of code rate, modulation order, and pilot arrangement. The performance comparison in this study provides useful information for industry engineers and standardization organizations developing future wireless technologies.
This paper is organized as follows. In
Section 2, the structures of single-carrier autoencoders and OFDM-based autoencoders are presented. The performance evaluation of single-carrier autoencoders and OFDM-based autoencoders in different channel models are shown in
Section 3. Finally, the conclusion of this study is given in
Section 4.
2. Application of Autoencoders for Wireless Communications
2.1. Single-Carrier Autoencoders
A general wireless communication system consists of separated blocks, illustrated in
Figure 1 [
15]. Each block in the system performs a distinct and independent function, such as channel encoding/decoding, digital modulation/demodulation, channel estimation, and equalization. This enables the wireless communication system to operate efficiently and flexibly. Indeed, many advanced techniques are used for the performance of each block individually [
16,
17,
18,
19,
20,
21]. Among these blocks, the channel encoding/decoding blocks play a crucial role to ensure transmission reliability under AWGN noise and frequency fading channels. Various schemes of channel coding are proposed, aiming to enhance the error-correcting ability while keeping minimal implementation complexity: Hamming codes, Reed–Muller codes, LDPC codes, and Polar codes. The LDPC codes are now widely used in popular advanced wireless communication systems such as 4G LTE, WiFi, 5G NR…
However, one of the major challenges in this system is optimizing the performance of individual blocks, which may not necessarily result in optimal performance for the entire system. To solve this issue, the proposed approach of using deep learning to optimize the entire system is being considered and researched. Deep learning is expected to improve performance better than conventional wireless communication systems.
An autoencoder is an artificial neural network (ANN) model primarily used for unsupervised learning tasks, especially in the field of deep learning. As shown in
Figure 2, the structure of an autoencoder is divided into an encoder and a decoder, which include the input layer, hidden layers, and output layer [
6]. An autoencoder can have one or multiple hidden layers that serve the encoding function to generate data that capture the most fundamental attributes necessary to fully describe the input data. Then, the decoder reconstructs an approximation of the encoder to generate an output that closely resembles the input. Autoencoders are considered unsupervised learning techniques because they do not require labeled data for training. However, to be more precise, we can use a form of self-supervision by creating our own labels from the training data.
The primitive function of AE is to transform data into another signal form, so that the new signal can be restored back to the most similar to the original data. In this study, we take advantage of AE to convert bit streams into complex signals, so AE can be meaningfully representative of the whole physical layer in wireless communications. However, a model that is too large will be challenging to train well. We proposed to train an AE model that can replace the channel encoder and modulator on the principle of ensuring fairness in comparison. More details on wireless autoencoder idea can be found in [
6].
The input bits to the autoencoder, denoted as vector x, are represented as a one-dimensional vector where the s-th element of the vector has a value of one and all other elements have a value of zero. That vector is called a one-hot vector, and is symbolized as . On the transmitter side, a neural network with multiple layers and a normalization layer ensures energy or power constraints on x. The receiver side also includes a similar neural network with corresponding layers as the transmitter. The final layer of the system utilizes the softmax activation function, producing an output vector , representing the probabilities of all possible characters. The decoded symbol s-th corresponds to the index of the element in (where i = 1, 2, …, M) with the highest probability. Afterward, the autoencoder can be trained end-to-end by using stochastic gradient descent (SGD) on the set of all possible symbols , utilizing the cross-entropy loss function to appropriately classify the difference between and p.
Regarding the training process, AE is trained by the SGD (stochastic gradient descent) algorithm combined with Adam. In it, each pair of training data consists of a symbol converted to a one-hot vector, as well as its probabilistic output after passing through the encoder, noise, and decoder. The loss function used is the following cross-entropy: Using the common backpropagation mechanism, the gradients are calculated, and the network layer weights at the encoder and decoder are updated through each loop in order to optimize the loss function. The visual results of the training process are shown in the next sections.
2.2. OFDM-Based Autoencoders
Recently, the OFDM technique has been an appropriate choice for wireless communications (such as 4G LTE and 5G NR mobile communications), since it is resilient in frequency-selective fading conditions and provides reliable synchronization ability. For that reason, the study in [
5] proposed a system utilizing OFDM modulation based on autoencoders. The block diagram of the system is illustrated in
Figure 3.
In contrast to the scheme of autoencoders for single-carrier modulation, a discrete Fourier transform (DFT) with a length of is applied to a set of symbols from the output of the autoencoder the length of , resulting in equivalent independent subchannels, where each symbol from the autoencoder is assigned to each subcarrier. To avoid intersymbol interference (ISI), a cyclic prefix (CP) of length is added, meaning that independent -encoded symbols (in the frequency domain) form a single OFDM symbol (in the time domain) with a total length of + samples. Therefore, a sequence consisting of ( + ) complex-valued symbols is transmitted through the transmission channel.
A structure of autoencoders in the OFDM communication system is presented in
Figure 4. In this system, the processed symbols are mapped to OFDM subcarriers after passing through a normalization layer and transmitted through the transmission channel. Two fully connected layers map
k (in the form of a one-dimensional network with length
M) to n real numbers. After the normalization layer, the OFDM modulation layer maps these
n real numbers to
complex symbols and assigns each symbol to a subcarrier. To ensure that the OFDM modulation layer outputs the complete set of OFDM symbols, the minimum input length is
. Thus, the input to the neural network is a sequence of one-hot values with a size of
.
4. Conclusions
In this study, we developed and evaluated the performance of wireless communication systems using single-carrier and OFDM-based autoencoders in a framework of the 5G NR system. The BLER performance of the system was considered compared with a conventional system with various modulation orders and LDPC code rates.
The simulation results show that the system with autoencoders provides superior performance compared to systems using LDPC channel encoding in low-signal-to-noise-ratio regions and high-modulation orders (64QAM and 256QAM). Specifically, a gain of 1–2 dB in signal power is obtained for single-carrier autoencoders and 0.3–2 dB is obtained for OFDM-based autoencoders. Since the complexity of implementing autoencoder schemes in communication systems is lower compared to designing LDPC encoding and decoding schemes, autoencoder is a promising candidate for enhancing wireless communications systems.
For further research directions, more optimal architecture of autoencoders should be investigated. Moreover, the impact of timing synchronization, frequency errors, and number quantization noise should be addressed.