Neural Network Equalisation for High-Speed Eye-Safe Optical Wireless Communication with 850 nm SM-VCSELs

Abstract

In this paper, we experimentally illustrate the effectiveness of neural networks (NNs) as non-linear equalisers for multilevel pulse amplitude modulation (PAM-M) transmission over an optical wireless communication (OWC) link. In our study, we compare the bit-error-rate (BER) performances of two decision feedback equalisers (DFEs)—a multilayer-perceptron-based DFE (MLPDFE), which is the NN equaliser, and a transversal DFE (TRDFE)—under two degrees of non-linear distortion using an eye-safe 850 nm single-mode vertical-cavity surface-emitting laser (SM-VCSEL). Our results consistently show that the MLPDFE delivers superior performance in comparison to the TRDFE, particularly in scenarios involving high non-linear distortion and PAM constellations with eight or more levels. At a forward error correction (FEC) threshold BER of 0.0038, we achieve bit rates of ~28 Gbps, ~29 Gbps, ~22.5 Gbps, and ~5 Gbps using PAM schemes with 2, 4, 8, and 16 levels, respectively, with the MLPDFE. Comparably, the TRDFE yields bit rates of ~28 Gbps and ~29 Gbps with PAM-2 and PAM-4, respectively. Higher PAM levels with the TRDFE result in BERs greater than 0.0038 for bit rates above 2 Gbps. These results highlight the effectiveness of the MLPDFE in optimising the performance of SM-VCSEL-based OWC systems across different modulation schemes and non-linear distortion levels.

1. Introduction

Optical wireless communication (OWC) stands as a pivotal technology that offers substantial opportunities to meet the demanding requirements of 6G and beyond and can serve as both access and cross-haul links; its versatile applications extend to device-to-device (D2D) communications and the Internet-of-Things (IoT) [1]. Two prominent OWC variants, visible light communication (VLC) and infrared OWC (IOWC), have showcased remarkable bit rates exceeding 20 Gbps for one optical wavelength [2], with VLC achieving up to 35 Gbps through wavelength division multiplexing (WDM) [3]. While VLC excels in scenarios requiring illumination, IOWC presents enhanced versatility and is particularly suitable for applications like LiFi uplinks and data centres. Integrating IOWCs with high-speed fibre communication systems has become increasingly significant, offering terabits-per-second (Tbps) indoor wireless access and interfacing seamlessly with fibre-to-the-home access networks [4]. Despite these advantages, the mobility of high-speed IOWC systems is constrained due to the use of narrow-divergence transmitters and receivers with narrow fields-of-view (FOVs). To address this limitation, implementing robust user tracking, localisation mechanisms, and optical beam steering is essential for the practical deployment of IOWC, thereby ensuring its effectiveness and widespread application [4,5].

Vertical-cavity surface-emitting lasers (VCSELs) stand out as important light sources for intensity-modulated/direct-detection (IM/DD)-based OWC systems, primarily owing to their distinct advantages over other optical sources. Notably, VCSELs boast cost-effectiveness in terms of fabrication and require lower electrical power consumption compared to edge-emitting lasers [6]. Their superiority extends to offering higher modulation bandwidths and superior emission coherence in comparison to light-emitting diodes (LEDs). Leveraging their vertical emission and streamlined fabrication, VCSELs can be seamlessly integrated into two-dimensional (2D) arrays, facilitating spatial diversity and enabling the deployment of multiple-input–multiple-output (MIMO) systems [7]. The significance of VCSELs is underscored by experimental endeavours in gigabit OWC, where individual VCSEL sources operating at wavelengths of ∼650 nm [8,9], ∼850 nm [10], and 1310 nm [2] have achieved bit rates of up to 25 Gbps using IM/DD. The pioneering fabrication of VCSELs within the 850–980 nm band marks a significant milestone in optical communications, and their continued dominance in the market is attributed to the alignment of these wavelengths with the high responsivity of cost-effective silicon photodetectors (PDs) [6]. However, the paramount consideration lies in ensuring the safety of the 850 nm system for the human eye, especially for protecting the retina. To meet the stringent class-1 laser safety requirements, compliance with the accessible emission limit (AEL) becomes imperative, and this is ∼−1.10 dBm, as specified by the International Electrotechnical Commission (IEC 60825-1) for a point source operating at 850 nm [11].

In the pursuit of optimising the information capacity of VCSEL-based OWC systems, extensive research has been dedicated to exploring spectral-efficient modulation schemes. Among these, multilevel pulse amplitude modulation (PAM-M) has emerged as a prominent contender. PAM involves transmitting symbols through pulses in a signal waveform, with the amplitudes of the pulses representing different symbols. A PAM transmission with M distinct amplitudes or “levels” represents ${log}_{2}M$ bits in each level, providing a robust mechanism for conveying information. PAM is the preferred choice for many commercial OWC systems due to its simplicity in handling real-valued symbols and its spectrum efficiencies that are comparable to other advanced modulation schemes such as orthogonal frequency division multiplexing (OFDM) [12]. However, PAM is not without challenges, mainly when deployed in high-speed communication scenarios, as it is susceptible to intersymbol interference (ISI) and system non-linearities [13]. Consequently, in numerous optical communication systems, PAM is often accompanied by an equaliser, such as a decision feedback equaliser (DFE) [13,14]. This strategic pairing ensures the mitigation of potential issues, enhancing the reliability and performance of VCSEL-based OWC systems employing PAM.

While DFEs effectively address ISI, non-linearity renders them inefficient, particularly for higher modulation levels. Therefore, non-linear DFEs like the Volterra-series equaliser and neural networks (NNs) are utilised to overcome ISI and non-linear effects. For instance, ref. [15] demonstrated that recurrent NNs outperform conventional DFEs in non-linear channels. Feedforward NNs, such as multilayer perceptrons (MLPs), radial basis functions (RBFs), and long short-term memory (LSTM), have been tested on optical channels using PAM-M [8,16,17]. Specifically, ref. [16] showed that MLP outperforms both conventional DFEs and Volterra-series equalisers.

To the best of the authors’ knowledge, few experimental studies have reported using 850 nm VCSELs with PAM-M for OWC transmissions, especially with launched optical powers below the AEL. The work outlined in [18] that uses 850 nm VCSELs reported a bit rate of 25 Gbps, but this was with a 7 × 7 array with an optical power of ∼22 dBm. The study in [8] employed a deep LSTM NN for effective ISI compensation in VCSEL-based OWC, achieving a data rate of 13.5 Gbps. However, the experimental work was limited to PAM-2, and the NN used at least 100 neurons per layer.

This paper presents the first experimental evaluations of directly modulated eye-safe single-mode VCSELs (SM-VCSELs) operating at 850 nm for high-speed OWC links. The evaluation incorporates PAM-M and employs an NN equaliser conducted over a 2.5 m OWC link. The SM-VCSEL is launched with a transmitted optical power of −1.47 dBm, which is securely below the AEL of −1.10 dBm, ensuring its adherence to the class-1 laser safety standards and thus complying with eye safety. Compared to multimode VCSELs (MM-VCSELs), SM-VCSELs offer advantages such as narrower spectral widths and lower-intensity noise [6]. We obtain a bit rate of about 29 Gbps at a link distance of 2.5 m with PAM-4 modulation and an NN equaliser. The main contributions of this work are summarised as follows:

• We show the first use of NN equalisers in an eye-safe 850 nm SM-VCSEL-based OWC that employs PAM-M schemes of up to 16 levels. The equalisers demonstrate superior performance to the conventional DFEs, particularly for high-level PAM {8,16}, which requires less bandwidth but suffers from higher device non-linearity. This is especially beneficial in OWC links using photodiodes with larger detection areas that offer higher power margins but lower bandwidths. While some studies have employed NN equalisers for OWC with PAM up to eight levels [8,17,19,20], this paper is, to the best of the authors’ knowledge, the first experimental work to achieve a multigigabit OWC link with PAM-16 and NN equalisers by leveraging the SM-VCSELs’ high signal-to-noise ratio (SNR).
• Previous work on NN equalisation for VCSEL-based OWC employed multiple hidden layers with over 100 neurons per layer. The NN equaliser for this study require less computational complexity, utilising only one hidden layer with six neurons.
• We highlight the non-linearity compensation capability of the NN equaliser by examining two modulation conditions for the VCSELs, with the first condition exhibiting lower non-linearity than the second. Notably, NN equalisers perform better in the second scenario. Additionally, for low-level PAM {2,4}, NN equalisers provide superior bit rate performance compared to DFEs with MM-VCSELs, despite MM-VCSELs having higher non-linearity than SM-VCSELs.

This study parallels the experimental report in [21], where we achieved a bit rate of 38 Gbps using optimised OFDM with bit and power loading. However, the goal of this work is to demonstrate the rates achievable using simple PAM modulation schemes and to experimentally show the effectiveness of NNs in compensating for non-linearity and ISI. While [21] aims to maximise the achievable bit rate through an optimised OFDM scheme, this work focuses on the non-linear behaviour of VCSELs and explores possible mitigation using an NN-based non-linear equaliser with multilevel PAM. Moreover, we employ lower modulation amplitudes in this study, as PAM has a lower peak-to-average-power ratio than OFDM.

2. Experiment Setup

Figure 1a depicts a photo of the VCSEL-based OWC system, showcasing its functionality over a transmission distance of 2.5 m. The essential instruments and devices governing the OWC system are highlighted in

Table 1. OWC system parameters.

Device/System	Module Description	Setup/Parameters
AWG	Keysight M8195A	BW = 23 GHz, SR = 60 GSa/s, 8-bit Res. DAC
Photodetector	Newport 818-BB-45A	BW = 9 GHz, ⌀ 60 $\mathsf{\mu }$m, ∼550 V/W Gain at 850 nm
Optics Lens	Thorlabs ACL2520U	⌀ 25 mm, Focal length = 20 mm
Oscilloscope	Tektronix DPO71254C	BW = 12.5 GHz, SR = 50 GSa/s, 8-bit Res. ADC

BW—Bandwidth, SR—Sampling Rate, and Res.—Resolution.

. Figure 1a is complemented by Figure 1b, which illustrates a schematic diagram presenting the PAM generation and decoding process employed within the IM/DD system. It begins with the offline generation of a random bit sequence using MATLAB, which is subsequently mapped to PAM symbols through gray coding. These PAM symbols undergo upsampling and processing via a non-return-to-zero (NRZ) rectangular pulse-shaping filter. The resulting signal is then loaded to an arbitrary waveform generator (AWG) for the creation of an analogue waveform, which, in turn, drives the VCSEL directly through a bias tee (Tektronix PSPL5542, 50 GHz).

The VCSEL output undergoes collimation using an aspheric lens (Thorlabs ACL2520U). At the receiver, the optical signal is collected through an aspheric lens and is focused onto a PIN PD module (818-BB-45A). To facilitate optimal OWC link alignment and receiver optical power adjustment, both the VCSEL and PD are mounted on 3D-axis stages (Thorlabs NanoMax 300). The received signal is then captured by an oscilloscope. Then, standard digital signal processing (DSP) techniques are employed, including synchronisation, digital filtering, and equalisation. Finally, the transformed PAM symbols are de-mapped into the received bitstream, facilitating the evaluation of the bit-error-rate (BER) performance.

The model that would best describe the output signal for the IM/DD OWC link assuming that most of the system non-linearity comes from the VCSEL is defined as [22,23]:

$$y\left(t\right)={\mathcal{G}}_{pd}{\mathcal{G}}_{owc}{f}_{L-I-V}\left((V_{bias}+s\left(t\right))\otimes h_{vcsel}\left(t\right)\right)\otimes h_{owc}\left(t\right)\otimes h_{pd}\left(t\right)+n\left(t\right),$$

(1)

where ${\mathcal{G}}_{pd}$ and ${\mathcal{G}}_{owc}$ denote the gains from the photodetector and OWC link, respectively; $f_{L-I-V}\left(.\right)$ denotes the non-linear transfer function that converts the input voltage signal to the optical signal; $V_{bias}$ denotes the bias voltage to drive the VCSEL, and $s\left(t\right)$ denotes the bipolar input voltage signal; $h_{vcsel}\left(t\right)$, $h_{owc}\left(t\right)$ and $h_{pd}\left(t\right)$ denote the impulse responses for the VCSEL, OWC link and photodetector, respectively; $n\left(t\right)$ denotes the noise. This memory-based model is particularly useful for high-speed OWC links with significant system non-linearities.

2.1. Single-Mode and Multimode VCSELs Comparison

The SM-VCSEL is custom-designed and fabricated by Integrated Compound Semiconductor (ICS) from a GaAs/AlGaAs-based epitaxial structure supplied by the Compound Semiconductor Centre (CSC), whereas the MM-VCSEL is an off-the-shelf device (OPTEK OPV310). Measurements characterising both devices, including light–current–voltage (L-I-V) curves, magnitude responses and noise power spectral densities (PSDs), can be found in [21]. The L-I-V curves show that the SM-VCSEL exhibits higher dynamic resistance and emits lower optical power due to its smaller oxide aperture, which ensures single-mode emission. To maintain eye-safe conditions, we drive the SM-VCSEL at approximately 2.2 mA, resulting in an optical power of around 0.7 mW (−1.47 dBm). Similarly, the MM-VCSEL is driven at about 5 mA to emit approximately 3 mW (4.7 dBm) of optical power. This ensures comparable bandwidths between the two VCSELs for meaningful performance comparisons. The measured −3 dB bandwidths of SM-VCSEL and MM-VCSEL are approximately 6.2 GHz and 6.4 GHz, respectively.

We compare both VCSELs for the OWC link based on their measured noise power spectral densities (PSDs) at a received optical power (ROP) of approximately −1.75 dBm. The noise PSDs for the SM-VCSEL and MM-VCSEL average around −137 dB/Hz and −129 dB/Hz, respectively. In comparison, the OWC system with a ROP of −1.75 dBm has thermal and shot noises estimated at −146 dB/Hz and −151 dB/Hz, respectively. Therefore, the primary noise source for the OWC link is the relative intensity noise (RIN) from the VCSELs, as the noise PSDs from both VCSELs exceeds that of shot or thermal noise.

We focus more on the eye-safe SM-VCSEL for PAM-M since it has a lesser RIN than the MM-VCSEL and offers a higher data rate for similar operating conditions [6,21]. To illustrate the performance of the equalisers for the SM-VCSEL, we consider two setups with the SM-VCSEL-based OWC link as follows:

• Setup-I: amplitude of 0.5 Vpp corresponding to ∼70% modulation index at 100 MHz, maximum ROP of −1.75 dBm;
• Setup-II: amplitude of 0.7 Vpp corresponding to ∼95% modulation index at 100 MHz, maximum ROP of −3.08 dBm.

While Setup-II offers a higher SNR than Setup-I, it does so at the expense of higher non-linearity. This is demonstrated in Figure 2, which illustrates the non-linearity of the SM-VCSEL (with both setups) and the MM-VCSEL (with Setup-I) through their total harmonic distortion (THD). The THD indicates the level of additional harmonics per frequency, with higher THD signifying greater non-linear distortion from these harmonics [24]. The MM-VCSEL exhibits approximately 3.5 dB more THD than the SM-VCSEL with both VCSELs at ∼70% modulation index. Moreover, the SM-VCSEL has about 2.2 dB more THD when Setup-II is used instead of Setup-I.

2.2. NN Equalisation

In channel equalisation tasks, various NN architectures can be employed [25]. However, our work uses multilayer perceptrons (MLPs), as they are among the oldest and the most common types of feedforward NNs, making them a good representation of other NNs [26]. MLP is a collection of perceptrons organised into layers, as illustrated in Figure 3. Each perceptron computes a linear combination of its inputs and incorporates an externally applied bias. The result of this combination is applied to an activation function [26]. The expression for the output of a perceptron is:

$$y=f_{avn}\left(w_{bias}+\sum _{i=0}^{m-1}{w}_i{x}_i\right),$$

(2)

where m denotes the number of inputs to the neuron, and $f_{avn}(.)$ is the activation function of the neuron; $x_i$ denotes the neuron inputs, $w_i$ denotes the weight for each input $x_i$, and $w_{bias}$ denotes the neuron bias.

A three-layer architecture suffices for the MLP for equalisation due to the universal approximation theorem [25,26]. This theorem asserts that for non-linear input–output mapping, if the activation function of neurons in the hidden layer is monotonically continuous, bounded and non-constant, a finite number of neurons in the layer can approximate the mapping effectively. The sigmoid function, which satisfies these conditions, is commonly employed as the activation function for neurons in the hidden layer. It is defined in (3) as:

$$f_{sgm}\left(x\right)={\displaystyle \frac{2a_1}{1+\exp (-a_{2}x)}}-a_1=a_{1}tanh\left(a_{2}x\right),$$

(3)

where $a_1$ and $a_2$ are suitably chosen constants. In line with the study in [16], tan-sigmoid ($a_1=a_2=1$) is employed in this study as the activation function. The activation function for the neuron in the output layer is the linear function ($f_{lin}\left(x\right)=x$). The feedback inputs can be sent to the MLP equaliser like with a conventional DFE, as illustrated in Figure 3. This MLP-based DFE (MLPDFE) can be expressed as [26]:

$${\tilde{z}}_n=\sum _{c=1}^{N_{hn}}\left[w_{c}tanh\left(w_{c,-1}+\sum _{a=0}^{N_{ft}-1}{w}_{c,a}{y}_{n-a}+\sum _{b=1}^{N_{bt}}{w}_{c,b}{\widehat{z}}_{n-b}\right)\right]+w_{-1},$$

(4)

where ${\tilde{z}}_n$ denotes the output of the MLPDFE; $\{y_n,\dots ,y_{n+1-N_{ft}}\}$ is the unequalised input sequence, and $\{{\widehat{z}}_{n-1},\dots ,{\widehat{z}}_{n-N_{bt}}\}$ is the set of previously detected symbols; $w_a$ and $w_b$ are the co-efficients of the feedforward and feedback tap weights, respectively; $N_{ft}$ and $N_{bt}$ denote the number of feedforward and feedback taps, respectively, for the DFE; $N_{hn}$ is the number of hidden-layer neurons; $w_{c,a}$ and $w_{c,b}$ are the synaptic weights for processing the feedforward and feedback inputs, respectively, for a hidden-layer neuron (with index c); $w_c$ denotes the weight used to process the neuron at the output layer; $w_{c,-1}$ and $w_{-1}$ denote the biases of a hidden-layer neuron and the output-layer neuron, respectively.

Compared to the conventional DFE, the MLPDFE requires more computational complexity and memory due to the additional hidden-layer neurons: each has a non-linear function. Based on the analysis done in [27], the MLPDFE needs about $N_{hn}$ times more real-valued multiplications than the conventional DFE. The performance of MLP is significantly impacted by the number of neurons in the hidden layer. Too few hidden neurons mean less computational complexity but could result in poor error performance. Conversely, an excessive number of neurons might lead to irregular error performance due to overfitting [25]. The back-propagation (BP) algorithm is popular for training the MLP equaliser. While there are multiple variants of the BP algorithm, the Levenberg–Marquardt BP (LMBP) algorithm is chosen in this study due to its superior convergence and better mean-square-error (MSE) performance [26]. However, LMBP uses batch training, which requires more computing memory than other BP algorithms that employ online training. The parameters for the MLP equaliser are highlighted in

Table 2. Equaliser parameters.

Parameter	Symbol	Value
Number of Forward Taps	$N_{ft}$	32
Number of Feedback Taps	$N_{bt}$	8
Number of Hidden-Layer Neurons	$N_{hn}$	6
Number of Training Symbols	$N_{tr}$	4000
Number of Bits for BER Testing		106

. Increasing the equaliser parameters beyond those in Table 2 offers negligible improvement at the cost of increased complexity.

3. Communication Performance and Analysis

To assess the effectiveness of the NN equaliser in handling PAM-M, this section presents the BER results from the SM-VCSEL-based OWC experiment setups (Setup-I and Setup-II) discussed in Section 2. The bit rates range from 2 Gbps to 40 Gbps in order to explore multigigabit communication over the SM-VCSEL-based OWC link. For comparison purposes, the link is evaluated (1) without equalisation (unequalised), (2) with the transversal DFE (TRDFE), and (3) with the MLPDFE. The parameters for both the TRDFE and the MLPDFE are summarised in Table 2.

The BER results are presented across various bit rates for the equalisers employing PAM-2 and PAM-4 schemes for the SM-VCSEL and the MM-VCSEL in Figure 4. A BER of ∼$3.8\times {10}^{-3}$, which is the second-generation super forward error correction (S-FEC) limit and has a 7% overhead and ∼6 dB coding gain [28], is used to estimate the system’s bit rate performance. For PAM-2, the MLPDFE output layer uses the sigmoid activation function due to its “binary” nature and superior performance compared to the linear activation function [16]. However, the linear activation function is better for PAM at four or more levels. Without equalisation at this FEC limit, the PAM-2 scheme with the SM-VCSEL achieves bit rates of ∼8 Gbps and ∼9 Gbps for Setup-I and Setup-II, respectively. Conversely, the bit rates achieved with PAM-4 are ∼5 Gbps and ∼4.5 Gbps for Setup-I and Setup-II, respectively. A notable observation is that Setup-II demonstrates superior performance with PAM-2, while Setup-II showcases better results with PAM-4. This discrepancy arises due to PAM-2’s lower susceptibility to non-linearity than PAM-4, leveraging the high SNR specifically achievable in Setup-II.

With PAM-2, the MLPDFE demonstrates a marginal BER performance improvement compared to the TRDFE across all bit rates. Both equalisers significantly enhance the OWC link, enabling bit rates of ∼23 Gbps and ∼28 Gbps at the 7% S-FEC limit for Setup-I and Setup-II, respectively. With PAM-4, the MLPDFE and TRDFE exhibit similar BER performances for bit rates exceeding 25 Gbps, achieving ∼27 Gbps and ∼29 Gbps at the 7% S-FEC limit for Setup-I and Setup-II, respectively. However, the MLPDFE notably outperforms the TRDFE for bit rates below 25 Gbps, i.e., at low BER values. For instance, at a BER of ${10}^{-6}$ with Setup-I, the MLPDFE achieves bit rates of ∼11 Gbps, while the TRDFE offers ∼6 Gbps. This is because non-linearity is the dominant performance-limiting factor at lower bit rates, while SNR becomes the limiting factor at higher bit rates. Hence, this result clearly demonstrates the effectiveness of the NN in compensating for non-linearity, which is further evident at higher modulation levels. It is worth mentioning that both equalisers exhibit better performance with PAM-4 on Setup-I than in Setup-II at bit rates below 25 Gbps. Specifically, the MLPDFE achieves ∼7 Gbps and ∼10 Gbps at a BER of ${10}^{-6}$ for Setup-I and Setup-II, respectively. Additionally, both equalisers showcase enhanced performance with PAM-2 on Setup-II relative to Setup-I at lower bit rates, as the MLPDFE attains ∼10 Gbps and ∼15 Gbps for Setup-I and Setup-II, respectively, at a BER of ${10}^{-6}$.

As with SM-VCSEL, equalisation significantly improves the performance of MM-VCSEL links. However, due to higher RIN from the MM-VCSEL, its achievable bit rates are considerably lower than those of SM-VCSEL at the 7% S-FEC limit, such as 15 Gbps with PAM-4 MLPDFE in Setup-I and ∼17.5 Gbps with PAM-2 MLPDFE with Setup-II. It is also worth noting that, unlike the SM-VCSEL, the MLPDFE offers better bit rate performance than TRDFE for PAM-2 from the MM-VCSEL because the MM-VCSEL has higher non-linearity than the SM-VCSEL. Nonetheless, we will focus only on the SM-VCSEL in the following sections due to its superior bit rate performance compared to the MM-VCSEL.

Figure 5 presents the BER results for bit rates with PAM-8 and PAM-16 using Setup-I (Figure 5a) and Setup-II (Figure 5b). Significant BER challenges are evident across different modulation schemes and setups without equalisation. Firstly, with PAM-8 modulation on Setup-I, the BER surpasses 0.07 for bit rates starting from 2 Gbps and beyond. Similarly, using PAM-16 on Setup-I exacerbates the situation, with the BER exceeding 0.15 for bit rates of 2 Gbps and higher. Conversely, with Setup-II, employing PAM-8 modulation results in a BER of ∼0.02 at a bit rate of 2 Gbps, suggesting relatively improved performance compared to Setup-I. However, with PAM-16 on Setup-II, the BER exceeds 0.09 for bit rates starting from 2 Gbps. These results underscore the critical importance of effective equalisation strategies in mitigating ISI and non-linearity for OWC links utilising PAM-8 and PAM-16.

The comparison between the TRDFE and the MLPDFE underscores the latter’s superior performance, particularly in Setup-II. With the TRDFE on Setup-I, the BER surpasses 0.03 for bit rates of 2 Gbps and beyond with PAM-8, escalating to over 0.09 with PAM-16. In contrast, Setup-II exhibits improved performance with the TRDFE but still encounters BER issues exceeding 0.07 for bit rates beyond 2 Gbps with PAM-16. These results demonstrate the ineffectiveness of the TRDFE to mitigate the ISI in the presence of non-linearity.

The MLPDFE, however, showcases remarkable improvements in both setups. In Setup-I, it achieves an impressive bit rate of about 22.5 Gbps at the 7% S-FEC limit with PAM-8, alongside a low BER of about $3.5\times {10}^{-4}$ at a bit rate of 3 Gbps with the same modulation scheme. Similarly, in Setup-II, the MLPDFE achieves a substantial bit rate of about 17.5 Gbps with PAM-8 at the 7% S-FEC limit, coupled with an exceptionally low BER of about ${10}^{-6}$ at a bit rate of 5 Gbps. These results signify the MLPDFE’s effectiveness in significantly improving transmission rates and minimising errors, which is particularly notable in Setup-II compared to the TRDFE, highlighting its superiority in optimising OWC systems with high non-linearities.

To highlight the effectiveness of the NN equaliser, eye diagrams for the output of each equaliser using PAM-8 and PAM-16 modulation schemes are presented in Figure 6. These diagrams are computed by upsampling the received PAM symbols by eight and filtering the result with a raised cosine filter with a 0.35 roll-off factor. The diagrams are presented at 12 Gbps for PAM-8 and 6 Gbps for PAM-16 in Figure 6a and Figure 6b, respectively. Without equalisation, the PAM-8 and PAM-16 waveforms exhibit severe distortion due to ISI and non-linearities, resulting in closed eye diagrams. Implementing the TRDFE improves the situation by opening the eye diagrams; however, the levels remain unequally spaced due to incomplete mitigation of system non-linearity.

In contrast, the eye diagrams with the MLPDFE show more uniform spacing between levels. This uniformity is attributed to the MLPDFE’s ability to compensate for the inherent non-linearity in the system, resulting in improved eye diagram characteristics. This improvement is also reflected in the BER plots in Figure 5, where the MLPDFE demonstrates superior performance compared to the TRDFE.

The results achieved with Setup-II were obtained at an ROP of ∼−3.1 dBm. However, misalignments and extensions in the OWC link can diminish the ROP, subsequently impacting the SNR and bit rate. Hence, Figure 7 is presented to illustrate the bit rate attainable at the 7% S-FEC limit for various ROPs ranging from −14 dBm to −3 dBm using PAM-2, PAM-4, PAM-8, and PAM-16 modulation schemes with both the TRDFE and MLPDFE. At lower bit rates and reduced ROPs, PAM-4 requires more power than PAM-2 to achieve similar bit rates. For instance, with the TRDFE at a bit rate of 10 Gbps, PAM-2 and PAM-4 necessitate ROPs of ∼−13.2 dBm and ∼−11.0 dBm, respectively, indicating an ROP difference of 2.2 dB in favour of PAM-2. However, at a higher bit rate of 25 Gbps, PAM-2 and PAM-4 demand ROPs of ∼−6.0 dBm and ∼−5.4 dBm, respectively, showcasing a reduced ROP difference of 0.6 dB between the two modulation schemes. PAM-4 offers a higher bit rate of ∼29 Gbps compared to PAM-2, which gives ∼28 Gbps at the high ROP of −3.1 dBm.

When considering PAM-2, the TRDFE shows a slight advantage over the MLPDFE at lower bit rates with reduced ROPs. However, as the bit rates increase with higher ROPs, the MLPDFE demonstrates superior performance over the TRDFE. For example, at a bit rate of 10 Gbps, the TRDFE and MLPDFE require ROPs of ∼−13.2 dBm and ∼−13.0 dBm, respectively, indicating a marginal ROP difference of 0.2 dB in favour of the TRDFE. However, at a higher bit rate of 25 Gbps, the TRDFE and MLPDFE demand ROPs of ∼−6.0 dBm and ∼−7.1 dBm, respectively, showcasing a notable ROP difference of 1.1 dB in favour of the MLPDFE.

With PAM-4, the MLPDFE demonstrates superior ROP performance compared to the TRDFE at lower bit rates with reduced ROPs. However, at higher bit rates with increased ROPs, the MLPDFE shows ROP requirements similar to those of the TRDFE. For instance, at a bit rate of 10 Gbps, the TRDFE and MLPDFE require ROPs of ∼−11.5 dBm and ∼−11.0 dBm, respectively, indicating a marginal ROP difference of 0.5 dB in favour of the MLPDFE. However, at a higher bit rate of 25 Gbps, both the TRDFE and MLPDFE demand an ROP of ∼−5.4 dBm, showcasing no ROP difference between the two equalisers.

However, increasing the modulation level to PAM-8 and PAM-16 does not improve the bit rate performance due to the limited SNR available in the system [29]. Consequently, no bit rates above 2 Gbps were achieved at the FEC limit with the TRDFE for these modulation levels. In contrast, the MLPDFE enables bit rates beyond 6 Gbps at ROPs greater than −10 dBm with PAM-8. Notably, with PAM-16, the MLPDFE achieves a consistent bit rate of ∼5 Gbps at ROPs ranging from −5 dBm to −3 dBm. Compared to PAM-8, which experiences a significant drop in bit rates with a decreasing ROP, PAM-16 shows a milder reduction. This is because PAM-16 requires lower bandwidth and thus experiences lesser noise at lower ROPs for bit rates similar to that of PAM-8.

In conclusion, the MLPDFE outperforms the TRDFE when the link SNR exceeds a certain threshold. This threshold is smaller with increasing PAM levels and higher system non-linearity, as indicated by the bit rate results from the two setups (Setup-I and Setup-II) using both VCSELs.

4. Conclusions

We have conducted experimental evaluations of an OWC system that uses an eye-safe 850 nm SM-VCSEL for a 2.5 m link. The system employs PAM-M in conjunction with an NN-based DFE, also known as the MLPDFE in this study. The SM-VCSEL offers distinct advantages, including lower RIN and lesser non-linearity, allowing for higher communication speeds compared to MM-VCSELs.

We compared the BER performances of the MLPDFE against that of the TRDFE for the SM-VCSEL setup under conditions of strong and weak non-linear distortions. Our results indicate that the NN-based equaliser consistently delivers superior performance across most investigated scenarios, particularly in VCSEL-based OWC systems with high non-linear distortion and PAM constellations of eight or more levels.

Specifically, the MLPDFE outperforms the TRDFE for PAM-4 modulation at lower ROPs, whereas it excels with PAM-2 at higher ROPs. With the MLPDFE, we achieved impressive bit rates of approximately 28 Gbps, 29 Gbps, 22.5 Gbps, and 5 Gbps using multilevel PAM schemes of 2, 4, 8, and 16 levels, respectively, at the 7% second-generation super FEC (S-FEC) limit. While higher modulation levels may not always offer the best bit rate because of their dependence on the available SNR, these results underscore the efficacy of the MLPDFE in optimising the performance of SM-VCSEL-based OWC systems across various modulation schemes and non-linear distortion levels.