Deep Learning Peephole LSTM Neural Network-Based Channel State Estimators for OFDM 5G and Beyond Networks

Abstract

This study uses deep learning (DL) techniques for pilot-based channel estimation in orthogonal frequency division multiplexing (OFDM). Conventional channel estimators in pilot-symbol-aided OFDM systems suffer from performance degradation, especially in low signal-to-noise ratio (SNR) regions, due to noise amplification in the estimation process, intercarrier interference, a lack of primary channel data, and poor performance with few pilots, although they exhibit lower complexity and require implicit knowledge of the channel statistics. A new method for estimating channels using DL with peephole long short-term memory (peephole LSTM) is proposed. The proposed peephole LSTM-based channel state estimator is deployed online after offline training with generated datasets to track channel parameters, which enables robust recovery of transmitted data. A comparison is made between the proposed estimator and conventional LSTM and GRU-based channel state estimators using three different DL optimization techniques. Due to the outstanding learning and generalization properties of the DL-based peephole LSTM model, the suggested estimator significantly outperforms the conventional least square (LS) and minimum mean square error (MMSE) estimators, especially with a few pilots. The suggested estimator can be used without prior information on channel statistics. For this reason, it seems promising that the proposed estimator can be used to estimate the channel states of an OFDM communication system.

1. Introduction

Deep learning (DL) significantly advances wireless communication systems’ concepts, patterns, methods, and means. Numerous interesting results have been for the physical layer or network layer of communications, including channel estimation (CE), channel state information (CSI) feedback compression, signal identification, and resource management. CE is one of the topics that has received the most research attention among all DL applications for wireless communication systems. The first attempt has been made to apply sophisticated DL algorithms to understand the properties of frequency-selective wireless channels and battle nonlinear distortion and interference for orthogonal frequency division multiplexing (OFDM) systems [1,2].

Wireless broadband systems often use orthogonal frequency division multiplexing (OFDM) to avoid selecting frequency-fading channels. For OFDM wireless communication systems to work well, the accuracy of the channel response estimation is very important. Pilots can derive information about the CSI before the transmitted information is detected. The receiver can retrieve the transmitted symbols based on the estimated CSI. Least square (LS) and minimum mean square error (MMSE) are the most popular methods for selecting the optimal estimation strategy. Despite its simplicity, the accuracy of the LS technique is often unsatisfactory. Due to the complex MMSE calculation, prior information on 2nd order channel statistics and noise variance is required [3]. This study proposes and implements a peephole long short-term memory (peephole LSTM) DL-based CSIE for OFDM wireless communication systems. To the authors’ knowledge, this work is the first to use the peephole LSTM DL network as a CSIE. The proposed estimator does not need prior knowledge of the communication channel statistics and powerfully works with limited pilots (under the condition of less CSI). The proposed estimator is data-driven to analyse, recognise, and understand the statistical characteristics of wireless channels suffering from many known interferences such as adjacent channel, inter-symbol, inter-user, inter-cell, co-channel, and electromagnetic interferences and unknown ones. Although an impressively wide range of configurations can be found for almost every aspect of DL, the choice of the loss function is underrepresented when addressing communication problems, and most studies and applications use the ‘log’ loss function.

The main contributions to this paper are:

• Propose and evaluate a DL-based peephole LSTM model for estimating channel state information and comparing MMSE, LS, and DL approaches for estimating the impulse response of different channels for next-generation 5G cellular mobile communications based on the OFDM system.
• Peephole LSTM designs can store data over a long period. This characteristic is extremely useful when we deal with time-series or sequential data. It is processing not only single data points (e.g., images) but also entire data sequences (such as speech or video inputs). It performs better than MMSE, with less complexity, and LS conventional-state estimators.
• The proposed estimator is trained for a generated OFDM signal, where the bit error rate (BER) is calculated and compared with the LS and MMSE estimations. During the offline training and the online deployment stages in this preliminary investigation, the wireless channel is presumed to be fixed. A random phase shift for each transmitted OFDM packet is implemented to assess the effectiveness of the neural network.
• This work uses three DL optimization techniques and the crossentropyex loss function to determine the most accurate and robust estimator with unknown channel statistical properties and a limited number of pilots.
• A comparative study was conducted with three DL optimization algorithms, namely SGDm, RMSProp, and Adam, on 64, 8, and 4 pilots, respectively, to investigate the efficiency of the proposed estimator. The suggested DL peephole LSTM estimator performance was compared with its competitors from previous works, LSTM [4] and gated recurrent unit (GRU) estimators [5].
• Examine the performance of the suggested framework for CE in various scenarios. The symbol error rate (SER) is specifically modelled to examine the accuracy of CE.
• The simulation’s final findings show the following:

  ✓

  The suggested estimator is superior to LS and MMSE, with fewer pilots (four and eight).

  ✓

  The SGDm model with 64 and 8 pilots performs worse than the Adam and RMSProp models.

  ✓

  The RMSProp model has a better SER with only four pilots than its traditional competitor.

Outlines of the article: Section 2 discusses related work. Section 3 introduces the channel estimation methods and system model. The results are discussed in Section 4. The paper concludes in Section 5.

2. Related Work

In recent years, numerous studies have shown that DL approaches are particularly well suited for CE, and DL is increasingly used in communication systems. Thus, this section reviews previous studies on our major issue.

Artificial neural networks (ANNs) attempt to mimic the behaviour of the human brain and are becoming increasingly popular today. Recently, ANNs, also referred to as feed-forward neural networks (FFNN) or recurrent neural networks (RNNs), have been used in CSI-based localisation [6,7], channel decoding [8,9], image recognition [10,11], user localisation, traffic information, service requests, and channel usage [8,9], and CE and recognition [9,10].

An FFNN-based combined symbol detection and CE approach was presented in [9] for the OFDM method with selectable frequency channels. When considering communication networks that are not perfect, the proposed algorithms perform better than the conventional estimators. An online DL-enabled estimator for double selectivity channels was proposed in [11]. Under all test conditions, the proposed algorithm outperformed the traditional linear MMSE estimators. In one paper [10], a model for 1D convolutional neural networks (CNNs) in DL was presented for estimating the channel and retrieving the equalized information. A comparison was made between the 1D CNNs and the FFNNs, MMSE, and LS estimators regarding bit error rate (BER) and mean square error (MSE) for various modulation schemes. The 1D CNN estimator outperformed its competitors. The ability of deep-learning algorithms to directly capture the features of a given problem without requiring prior knowledge is a significant advantage for channel estimators. RNNs have not been a standard network system in recent years due to the challenges associated with training and computational complexity. Due to the growth of DL theory, RNNs have just entered a phase of rapid development. RNNs are currently successfully used in speech and handwriting recognition [12,13]. However, the vanishing/exploding gradient problem affects the basic RNN. It also shows how difficult it is for RNNs to learn and maintain long-term memory and that they are limited in their ability to influence data. Long-short-term memories (LSTMs) were suggested to overcome the vanishing gradient issue in [4,14,15,16].

The paper [17] presented a hybrid model and data-driven receiver technique that utilizes LS estimation as well as zero-point equalization to take out the initial features for estimating channels and identifying data. The paper [18] achieved reliable CE using an upgraded extreme learning machine (ELM). LS highlights initial linear characteristics for CE in the model-driven mode. An upgraded ELM network is built using the initial features to refine channel estimates. There are numerous improved variants of LSTM in its historical evaluation, such as BiLSTM, ConvLSTM, GRU, and peephole long short-term memory (peephole LSTM), which offer the best performance in some specialized areas. A bidirectional long short-term memory (BiLSTM) network comprises two LSTMs with forward and backward propagation, and it may concurrently capture forward and backward features [19,20,21]. Convolutional long-short-term memory (ConvLSTM) uses the convolution operator to calculate a cell’s future state, which is then determined by its neighbours’ inputs and previous states [22].

The GRU is a simpler LSTM variant. The GRU architecture used to solve RNN problems with LSTM has a rather complicated structure [5]. A. S. M. Mohammed et al. [23] compared DL-based CE to least-square (LS) and minimum mean-square error (MMSE) estimators in pilot-assisted channel estimation (PACE). A GRU-based DL strategy is used to extract time and frequency domain features for CE to suppress error propagation of the data pilot-aided process applied to IEEE 802.11p in [24]. In the letter [25], DL helps OFDM recover nonlinearly corrupted transmission data. A self-normalizing network (SNN) for CE, a convolutional neural network (CNN), and a bidirectional gated recurrent unit (BiGRU) for signal identification have been proposed.

3. CE Methods and System Model

CE is an important technique in OFDM architecture [26]. CE is explicitly defined as the description of a mathematically modelled channel. CE Algorithms are usually used to determine the channel impulse response or frequency response. Figure 1 shows the overall concept of CE. Channel estimator specification requirements include minimization of mean square error (MSE) and computational complexity. CE can be classified into PACE, blind channel estimation (BCE), and decision-directed channel estimation (DDCE). PACE is the most common method of transmitting a known signal from a transmitter, where pilot means the reference signal used by both a transmitter and a receiver. This can be extended to any wireless communication system and is of very low computational complexity. The key drawback, however, is the decrease in the transmission rate because non-data symbols (pilots) are added. One design challenge for PACE is thus to jointly minimize the pilot numbers while precisely estimating the channel.

The current research focuses on reaching and evaluating CE techniques such as the proposed DL-based peephole LSTM model, DL-based GRU model, and DL-based LSTM model for 5G channel models using PACE. The proposed method’s performance will be compared to the LS and MMSE estimations.

3.1. Proposed Method

This section presents a DL-based peephole LSTM model for channel state estimation. Another popular LSTM alternative with an associated connection, namely the peephole connection that the cell requires to control the gates. As the gates of the LSTM cells do not have direct connections from the cell state, which has only been possible through an open output gate, the network’s performance is affected by a lack of essential information. Gers and Schmidhuber have connected LSTM memory cells with peepholes from cell to gates. Also, peephole connections allow gates to use the previous internal and hidden states (which the LSTM cell is limited to). This allows peephole LSTM to learn more precise timings than traditional LSTM cells. Peephole connections have several advantages over other types of connections in that they can connect the associated gates of the current information of the memory cell at any time since they can observe the state of the memory cell before its modulation by the output gate. Peephole connections boost LSTM performance greatly, leading to their adoption as a typical application technique [27,28,29].

The peephole LSTM functions similarly to the standard LSTM, except for the computations for the peephole connections. Figure 2 demonstrates a peephole LSTM structure implemented as follows: There are several variants of the basic LSTM architecture. By making peephole connections, the gates can rely not only on the input layer $x_t$ and the past hidden state $h_{t-1}$, but also on the preceding internal state $c_{t-1}$, which adds another term to the gate equations that also comes back from the cell $c_t$, and the forget gate $f_t$ is generated by the activation function ${\delta }_g$. The forgetting gate’s output, which represents the capability for forgetting to hide the state of the cell in the top layer, is evaluated using the sigmodal’s output, which has values between 0 and 1, for which the mathematical formulation is given below:

The mathematical expression is:

$$f_t={\delta }_g(W_f{x}_t+V_f{h}_{t-1}+U_f{c}_{t-1}+b_f)$$

(1)

The actual series position’s input is processed by the input gate, which has three elements; the cell state, input data, and peephole connection value. When the sigmoid function is activated, the output is $i_t$:

$$i_t={\delta }_g(W_i{x}_t+V_i{h}_{t-1}+U_i{c}_{t-1}+b_i)$$

(2)

Next, combine two elements to update the cell state. The first element is generated by multiplying $c_{t-1}$ and the forget gate’s output, while the second part is generated by the activation function’s output of $x_t$, $c_{t-1}$, $h_{t-1}$ by an activation function. In this way, the updated cell state is given:

$$c_t=i_t{\delta }_c\left(W_c{x}_t+U_f{c}_{t-1}+V_f{h}_{t-1}+b_i\right)+c_{t-1}{f}_t$$

(3)

The following information affects the output gate’s decision: input value, most recent cell state, and peephole value. Memory also processes the new cell state. The sigmoid function determines the linked cell state and output gate value.

$$o_t={\delta }_g(W_O{x}_t+U_O{c}_{t-1}+V_O{h}_{t-1}+b_O)$$

(4)

To update the hidden state by multiplying the output gate and the cell state activated by the function.

$$h_t=o_t{{\delta }_h(c}_t)$$

(5)

The output of the entire LSTM model is as follows:

$$y_t=k(W_h{h}_t+b_y)$$

(6)

where $W_f$, $W_i$, $W_o$, $V_f$, $V_i$, $V_o,$ and $u_f,u_{i,}{u}_o$ are weight matrices, $W_h$ represents the hidden output weight matrix, and $b_f,b_i,b_o,b_y$ are bias vectors.

The DL peephole LSTM model was used to estimate the channel constructed from five layers and is presented below. The input data properties have a 256-bit serial input layer. The subsequent layer is a peephole LSTM layer having 16 hidden units that produce the final element of the sequence. A fully connected 4-class layer of SoftMax and a classification output layer follow. The peephole LSTM model can be deepened by adding further layers. Figure 3 illustrates the design of the suggested channel estimator.

When training the suggested DL peephole LSTM model, three optimization methods are applied for the loss function’s minimization: adaptive moment estimation (Adam), root mean square propagation (RMSProp), and stochastic gradient descent with momentum (SGDm). Due to the limited number of pilots, we have been searching for the most accurate and reliable estimator possible.

Recent research has led to the developing of a series of CSI channel models that accurately reproduce the actual channel’s statistics. These channel models can effectively model training data. In this study, we used the 5G channel model. According to TR38.901 [30,31], this model describes errors affecting CE performance. Other channel types include narrowband Rayleigh fading channels and double selective fading channels [11].

The training dataset for one subcarrier is randomly generated during offline training. Transmitters provide OFDM frames to receivers over the adopted channel. After that, online training tracks channel conditions to recover transmitted data. The OFDM signal is recovered using transmitted frames subjected to various channel defects, including channel distortion and noise. Three optimization methods train the suggested estimator in simulated datasets to produce the best model with the fewest pilots. The training dataset for a subcarrier is collected. The transmitter sends data and pilot symbols in each OFDM packet by multiplexing pilot series data symbols. The offline process and training symbol creation shown in Figure 4 is used to create the proposed peephole LSTM-based channel estimator. The flowchart of the proposed model is shown in Figure 5.

3.2. DL-Based LSTM Network for CSE

Among the most significant advantages of the long-term memory network (LSTM) are storing long-term dependencies and using previous calculations [32]. The LSTM consists of three gates: The input gate regulates when memory cells are activated, the forget gate helps the network clean the memory cells and forget the previous input data, and the output gate converts part of the cell state to a hidden state. In previous work, we proposed an LSTM-based CS estimator. The suggested DL peephole LSTM network was compared to previous studies. The article [4] discusses the DL LSTM-based CS estimator for OFDM wireless communication systems.

3.3. DL-Based GRU Network for CSE

GRUs are a gating approach in RNNs [33]. GRUs are LSTMs with a forgetting gate but have fewer parameters than LSTMs because they lack an output gate. Figure 6 shows the architecture of the GRU. The update gate, a recent addition to this architecture, combines the forget and input gates. Unlike the LSTM network, which contains three gates, the GRU network has only two (reset gate r and update gate z). The memory state cell included in the LSTM model is also absent from the GRU. As a result, the GRU architecture is one gate and memory state cell smaller than the LSTM. In previous work, we proposed a GRU-based CS estimator. We compare the results of the proposed DL peephole LSTM neural network with those of the previous research. This study [5] details the DL GRU-enabled CS estimator for OFDM wireless communication systems.

3.4. System Model

The architecture of the OFDM techniques and the OFDM baseband network is depicted in Figure 7. A serial-to-parallel converter is required at the transmitting edge to transfer the transmitted symbols and pilots into a parallel data flow. Inverse discrete Fourier transform (IDFT) converts frequency to time bands. Intersymbol interference (ISI) must be minimized using the cyclic prefix (CP). Channel delay spread should not exceed the CP duration. Complex stochastic variables determine the sampling space’s multipath channel (MPC). {b(n)}_(n = 0)^(N − 1), can be used to describe the received message signal, as in [5]. Thus, the following equation expresses the received signal y(n):

$$y\left(n\right)=x\left(n\right)\oplus b\left(n\right)+w\left(n\right)$$

(7)

where x(n) is the transmission signal, w(n) is additive white Gaussian noise (AWGN), and ⊕ is the periodic convolution. After implementing DFT and removing the CP, the receiving signal within the frequency band:

Y(k) = X(k)H(k) + W(k)

(8)

where correspondingly Y(k), X(k), H(k), and W(k) are the DFTs of y(n), x(n), b(n), and w(n).

The initial OFDM block contains pilot symbols, while successive blocks contain transmitted data to create a frame. Although the channel changes between the different frames, it can be considered stationary over one frame. The proposed DL peephole LSTM-based CSE receives information and restores the sent information at its output.

4. Simulation Results

This section discusses the DL peephole LSTM-based CS estimator proposed for OFDM 5G and beyond networks. Then, the suggested estimator is compared with the LSTM and GRU-based CS estimators. The proposed estimator, LSTM and GRU were trained on simulated datasets and their symbol error rates (SERs) compared to the conventional LS and MMSE CSE estimators under different signal-to-noise ratios (SNRs).

The training dataset for a subcarrier was collected. The transmitter transmitted data and pilot symbols in each OFDM packet. Multiplexing the pilot series data symbols, there were 10,000 OFDM packets generated, with 80% at training and 20% at validation. The proposed estimator, LSTM, and GRU CSEs all follow the estimator settings and training options in

Table 1. Channel state estimators’ structure and training parameters.

Parameter	Value
Input data size	256
Layer hidden units	16
Fully connected layer size	4
Loss function	Crossentropyex
Mini batch size	1000
Number of epochs	100
Optimization algorithm	RMSProp, Adam and SGDm
Training data size	8000—OFDM frame
Validation	2000—OFDM frame
Testing data size	10,000—OFDM frame

. On the other hand,

Table 2. Parameters for the OFDM system’s channels.

Parameter	Value
Modulation mode	Quadrature phase-shift keying
Carrier frequency	2.6 GHz
No. of paths	24
Cyclic prefix length	16
No. of subcarriers	64
No. of pilots	4, 8, and 64

shows the OFDM channel parameters. In addition, in the actual simulations, the proposed estimator, the LSTM, and the GRU were trained with ADAM, RMSProp, SGDm, and optimization algorithms to investigate their performance under these learning approaches.

Figure 8, Figure 9, Figure 10 and Figure 11 illustrate the effects of the number of pilots by comparing the proposed estimator with the LS and MMSE estimators. Using a variety of pilots of 4, 8, and 64 and assuming identical channel conditions and the Adam optimizer, the performance of the peephole LSTM-based CSE, LS, and MMSE estimators was evaluated.

Figure 8 indicates that the suggested estimator is superior to the LS estimator at SNRs from 0 to 18 dB and achieves identical results to the MMSE estimator at SNRs from 0 to 10. The proposed CSE strongly competes with the MMSE CSE at low SNRs. The MMSE estimator achieved better results than the LS estimator at all SNR values. The MMSE method incorporates Gaussian noise and employs second-order channel statistics to estimate performance.

Figure 9 and Figure 10 indicate that the DL peephole LSTM-based estimator was superior to the LS and traditional MMSE estimators with fewer pilots.

As shown in Figure 9, using eight pilots led to a degradation of the estimators LS and MMSE, while Figure 10 shows that LS and MMSE became useless at 0 dB when only four pilots were used. On the other hand, the proposed estimator can further decrease its SER when the SNR increases. This proves that the DL peephole LSTM-based CSE was not affected by the few pilots available for CE.

As seen in Figure 8, Figure 9 and Figure 10, the LS estimator performed the worst because it did not use primary channel data in the estimating method. Nevertheless, the MMSE estimator performed better using second-order channel statistics during the estimation process, especially when many pilots were available.

Figure 11 shows how the suggested estimator performed for pilot numbers 4, 8, and 64. At 64 pilots, the DL peephole LSTM channel estimator outperformed all others. Thus, more pilots improved the suggested estimator’s performance.

Figure 12, Figure 13, Figure 14, Figure 15, Figure 16, Figure 17 and Figure 18 show the effects of the different optimization algorithms on system performance. The three optimization methods, RMSProp, SGDm, and Adam, are discussed and compared in this part. Next, we will investigate how well the SGDm and RMSProp learning techniques performed in obtaining a more accurate DL peephole LSTM-based channel estimator.

Choosing the best approach for optimizing a particular scientific problem can be difficult. The learning process is not improved if the network remains at the local minimum during training due to an inaccurate optimization technique. DL peephole LSTM-based CE optimization requires a comparison of model-based and dataset-based optimizers.

Figure 12 shows how the Adam model outperformed its competitors at 64 pilots. Figure 13 and Figure 14 show how the RMSProp models outperformed the Adam and SGDm models at eight and four pilots, respectively. The same optimizer performed differently depending on how many pilots it had.

As we can see in the end, the SGDm models performed the worst of all the cases. All parameters of the SGDm models had one learning rate.

As can be seen in Figure 15, the additional optimization needs of the suggested DL peephole LSTM estimator and its robustness when used with constrained pilots. The proposed estimators can be used with lower pilots at low SNRs (0–4 dB)—consequently, the transmission rate increases.

Tracking the DL training process is helpful, and the loss measurements help us evaluate training. As shown in Figure 16, Figure 17 and Figure 18, the trained DL peephole LSTM-based CS estimators using the SGDm algorithm provided poor performance compared to the Adam and RMSProp optimization strategies. The loss of the Adam and RMSProp optimization methods can be seen in Figure 12, Figure 13 and Figure 14.

Figure 19, Figure 20 and Figure 21 show the comparison of the effects of the different pilots on peephole LSTM, LSTM, and GRU-based CSEs. In this part, the results from the previous subsection are compared with those of the LSTM and GRU-based CSEs. To evaluate the performance of the three estimators under identical channel conditions, pilots 4, 8, and 64 were used. In this simulation, the RMSProp optimizer was used.

Figure 19 shows that for 64 pilots, the DL peephole LSTM-based estimator beat the rest of the studied CSEs over the entire SNR study range (0–20 dB). The performance of SNRs between 0 and 14 dB was identical to LSTM- and GRU-based CSEs.

The peephole LSTM and the LSTM performed better than the GRU for all SNR values. LSTM CSE outperformed peephole LSTM and GRU, as shown in Figure 20. At low SNRs, from 0 to 5 dB, the proposed CSE performed comparably well to the estimators of LSTM and GRU. The LSTM estimator was superior to the GRU estimator at all SNR values.

Figure 19 and Figure 20 demonstrate that the DL peephole LSTM-based estimator outperformed the GRU estimators when the number of pilots rose (8 and 64). In the case of using four pilots, Figure 21 shows that the peephole LSTM performed comparably to the LSTM and GRU estimators at SNRs between 0 and 9 dB. The GRU CSE outperformed the peephole LSTM and the LSTM in the remaining SNR test range (9–20 dB).

Table 3. The accuracy percentage for the suggested estimator vs. the other estimators.

Number of Pilots	64	8	4
Peephole LSTM	99.95%	99.41%	99.65%
LSTM	99.78%	99.55%	99.75%
GRU	99.88%	87.78%	99.97%
LS	99.98%	91.54%	0.08%
MMSE	100%	91.25%	0.20%

summarizes the accuracy percentages of the proposed and traditional estimators. The tabulated results emphasize the above-presented results. It can be noted that the proposed and its peers from deep learning-based CSEs outperformed the traditional LS and MMSE estimators when using fewer pilots (4 and 8).

5. Conclusions

This study proposes and evaluates a DL peephole LSTM model-based CSI estimator. The proposed model provides outstanding learning and generalization capabilities. It also can improve future communication systems, including 5G. The suggested estimator is trained offline and then applied online to monitor channel statistics in the real-life communication system so that channel information can be approximated, and hence the transmitted data can be recovered successfully. The performance of the suggested estimator was evaluated in a comparative study using Adam, RMSProp, and SGDm, three different DL optimization techniques, and different pilots (64, 8, and 4). The suggested estimator outperformed LS and MMSE estimators when just a few pilots were available. The results demonstrate that the SGDm model with 64 and 8 pilots performs worse than the Adam and RMSProp models. The RMSProp model has a better SER with only four pilots than its traditional competitors.

Using RMSProp at different pilots, 4, 8, and 64, the peephole LSTM-, LSTM-, and GRU- based CSEs were compared. The results demonstrate that the peephole LSTM outperforms GRU when the number of pilots rises (8 and 64). With four pilots, GRU outperforms the peephole LSTM and LSTM for high SNRs.

The suggested estimator is promising for OFDM 5G communication system channel state estimation with few pilots without the prior channel knowledge required. For future work, the authors suggest the following research plans: Examine the efficiency and accuracy of the suggested estimator using various cyclic prefix kinds and lengths. Evaluate the performance of CNN, GRU, and peephole LSTM-based CSIEs utilizing robust loss functions with robust statistical estimators like M-estimators and traditional loss functions with traditional statistical estimators like crossentropyex, MAE, and SSE.