Research on Orbital Angular Momentum Multiplexing Communication System Based on Neural Network Inversion of Phase

Abstract

An adaptive optical wavefront recovery method based on a residual attention network is proposed for the degradation of an Orbital Angular Momentum multiplexing communication system performance caused by atmospheric turbulence in free-space optical communication. To prevent the degeneration phenomenon of neural networks, the residual network is used as the backbone network, and a multi-scale residual hybrid attention network is constructed. Distributed feature extraction by convolutional kernels at different scales is used to enhance the network’s ability to represent light intensity image features. The attention mechanism is used to improve the recognition rate of the network for broken light spot features. The network loss function is designed by combining realistic evaluation indexes so as to obtain Zernike coefficients that match the actual wavefront aberration. Simulation experiments are carried out for different atmospheric turbulence intensity conditions, and the results show that the residual attention network can reconstruct the turbulent phase quickly and accurately. The peaks to valleys of the recovered residual aberrations were between 0.1 and 0.3 rad, and the root means square was between 0.02 and 0.12 rad. The results obtained by the residual attention network are better than those of the conventional network at different SNRs.

1. Introduction

Orbital Angular Momentum (OAM) has become an object of research as one of the means to improve the transmission efficiency of optical communication [1]. Recently, stable data transmission at the Tbit/s level has been achieved in the laboratory by multiplexing OAM beams [2]. OAM multiplexed communication can be combined with polarization multiplexing and wavelength division multiplexing to enhance the transmission rate [3]. The transmission rate can be further improved by combining OAM multiplexing with Multiple Input Multiple Output (MIMO) spatial multiplexing techniques [4]. In addition, for OAM multiplexing-based Free Space Optical Communications (FSO) systems, MIMO equalization techniques can be utilized. A constant-mode blind equalization algorithm is used to improve the BER degradation in OAM multiplexing systems due to atmospheric turbulence [5]. It can also be combined with optimal mode selection strategies to ensure the communication quality at high-speed communication [6]. However, in FSO systems, the atmospheric refractive index variations due to atmospheric turbulence cause random amplitude undulations as well as phase distortions in the atmospherically transmitted laser beam, resulting in degradation of the performance of the FSO system [7].

Adaptive Optics (AO) technology can achieve an improved performance of communication systems by correcting the dynamic wavefront aberrations of the beam [8]. In AO, wavefront-free detection AO systems can directly use the light intensity information to design control algorithms that generate the control signals required by the wavefront corrector, i.e., wavefront reconstruction based on the aberrated light intensity image [9]. Classical wavefront reconstruction methods for wavefront-free detection include the Gerchberg–Saxton (GS) phase recovery algorithm, stochastic parallel gradient descent algorithm, simulated annealing algorithm, and genetic algorithm [10], which mostly need to be solved by iterative computation and are difficult to achieve real-time wavefront reconstruction.

Recently, deep learning techniques have been widely used in AO. Deep learning-based wavefront-free detection aims to use the light intensity image captured by the charge coupled device (CCD) camera as the input of a neural network and the wavefront aberration or Zernike coefficient as the output; then, the output is transformed into the control signal, and finally, the deformation mirror is controlled to achieve wavefront correction. Paine et al. [11] applied a deep learning approach to the point expansion function to achieve wavefront reconstruction. Tian et al. [12] recovered the wavefront based on a deep neural network model to address the problem of too many iterations of existing search algorithms. These studies showed the effectiveness of deep neural network models, and to reduce the computational complexity, Nishizaki et al. [13] estimated wavefront Zernike coefficients directly from a single light intensity image by using a convolutional neural network model and improved the network estimation performance by preprocessing. Zhai et al. [14] also used a convolutional neural network model to reduce the computation time. Ma et al. [15], inspired by the phase difference approach, proposed to use the light intensity maps in the focal and out-of-focus planes as the input to the neural network and output of the wavefront Zernike coefficients. Ma et al. [16] also explored the effect of the consistency of the training data on the wavefront recovery performance. Wu et al. [17] also used a convolutional neural network to establish a mapping of the real image to its Zernike coefficients based on the idea of phase difference. Zhang et al. [18] used residual networks to obtain wavefront Zernike coefficients and verified the robustness of deep residual networks.

The above literature shows that deep convolutional neural networks have shown better performance than traditional wavefront detection algorithms in wavefront-free detection AO systems, but they still suffer from inadequate perception and representation of light spot features. As the number of network layers deepens, gradient disappearance, explosion and overfitting are prone to occur, making it difficult to accomplish real-time accurate reconstruction tasks. Too many pooling operations can also lead to the loss of small target information, resulting in excessive reconstruction errors.

2. Vortex Beam Atmospheric Transport Model

The schematic diagram of the OAM state multiplexed communication system with wavefront correction is shown in Figure 1. The system starts with n path independent messages transmitted using binary signals, and then, it passes through an optical modulator for baseband signal modulation, modulating each user’s message onto a Gaussian optical carrier and then loading the OAM mode using different OAM mode converters. The next n path OAM state signals are multiplexed by optical intensity superposition and distorted by optical intensity distortion after atmospheric turbulence. To reduce the effects of mode crosstalk and turbulence, wavefront correction techniques are introduced to improve the received optical quality. Afterward, the OAM light is transformed back to Gaussian light by OAM state demultiplexing and inverse converters, and finally, the signal is demodulated into a binary signal by an optical demodulator.

In free-space optical communication, the vortex beam has a continuous spiral wavefront compared to an ordinary beam, and the phase has uncertainty in the direction of beam propagation, where the phase singularity is. The Laguerre–Gauss (LG) beam in the vortex beam is a representative beam, and its light field expression in free space is shown in the following equation [19].

$$\begin{array}{l}{u}_p^l(r,\phi ,z)=\sqrt{\frac{2p!}{\pi (p+\left|l\right|)!}}\frac{1}{w(z)}{\left[\frac{\sqrt{2}}{w(z)}\right]}^{\left|l\right|}\times \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{L}_p^{\left|l\right|}\left[\frac{2r^2}{w^2(z)}\right]\exp \left[\frac{-r^2}{w^2(z)}\right]\times \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\exp (-il\phi )\exp \left[\frac{ik{r}^{2}z}{2(z^2+z_0^2)}\right]\times \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\exp \left[-i(2p+\left|l\right|+1)\arctan \left(\frac{z}{z_0}\right)\right]\end{array}$$

(1)

where $p$ denotes the radial index of the LG beam; $r$ is the radial distance from the spatial point to the transmission axis; $\phi$ is the azimuth angle; $w(z)=w_0\sqrt{1+{(z/z_0)}^2}$, $w_0$ is the beam waist radius; $L_p^{\left|l\right|}$ is the Laguerre polynomial, $l$ is the topological charge of the vortex beam, which can be taken as an integer; $k=2\pi /\lambda$ is the wavenumber; and $z_0=k{w}_0/\lambda$ is the Rayleigh distance.

When the light wave passes through multiple phase screens, only the phase changes while the amplitude remains the same. Under the Rytov approximation, the expression for the optical field at a transmission distance of $\Delta z$ can be obtained, as shown in the following equation [20].

$$u(r,\phi ,z_{i+1})={ℱ}^{-1}\left\{ℱ\left\{u(r,\phi ,z_i)\exp \left[i\varphi \left(r,z_i\right)\right]\right\}\exp \left[\frac{-i\left(k_x^2+k_y^2\right)\Delta z}{2k}\right]\right\}$$

(2)

where ${ℱ}^{-1}$ and $ℱ$ denote the Fourier inverse transform and Fourier transform, respectively; $\varphi \left(r,z_i\right)$ denotes the atmospheric turbulence phase screen function; $k_x$ and $k_y$ denote the number of spatial waves in $x$ and $y$ directions, respectively, and $k=\sqrt{k_x^2+k_y^2+k_z^2}$.

Therefore, only the phase screen model of atmospheric turbulence needs to be simulated to calculate the magnitude of the optical field at the receiving end. In this paper, the Zernike polynomial defined by Noll is chosen to model the turbulent phase screen $\varphi (r)$, which can be expressed by the following equation.

$$\varphi (r)={\displaystyle \sum _i{a}_i{Z}_i}$$

(3)

$$\{\begin{array}{c}{Z}_{eveni}=\sqrt{n+1}{R}_n^m\left(r\right)\sqrt{2}\cos \left(m\theta \right),m\ne 0\\ Z_{oddi}=\sqrt{n+1}{R}_n^m\left(r\right)\sqrt{2}\sin \left(m\theta \right),m\ne 0\\ Z_i=\sqrt{n+1}{R}_n^0\left(r\right),m=0\end{array}$$

(4)

$$R_n^m\left(r\right)={\displaystyle \sum _{s=0}^{\left(n-m\right)/2}{\left(-1\right)}^s\frac{\left(n-s\right)!}{s!\left[\left(n+m\right)/2-s\right]!\left[\left(n-m\right)/2-s\right]!}{r}^{n-2s}}$$

(5)

where $r$ and $\theta$ are polar coordinates. $n$ and $m$ are the radial and angular steps, respectively; they are always integers and satisfy the following condition: $m\le n,n-\left|m\right|=even$.

The first term of the Zernike polynomial represents the translation, which has no effect on the image imaging quality. Higher-order polynomials, which can represent higher-frequency components, are therefore disregarded in this paper, and a polynomial of order 36 is selected to generate a realistic atmospheric turbulence phase screen.

Denoting the ith input signal as $x_i(t)$, each signal will piggyback on the corresponding OAM mode, so the output signal of each OAM state is denoted as $X_i^{Tx}=x_i(t)u_i(r,\phi ,z)$. The n-way signal multiplexing yields the following equation.

$$X_{mux}^{Tx}={\displaystyle \sum _{i=1}^n{x}_i(t)u_i(r,\phi ,z)}$$

(6)

After atmospheric transmission, assuming that the mode crosstalk and noise are independent of each other during transmission, the received signal can be expressed as the following equation.

$$Y_{mux}^{Tx}={\displaystyle \sum _{i=1}^n{x}_i(t)u_i^{’}(r,\phi ,z)+n(t)}$$

(7)

where $u_i^{’}=u_i\times \exp (j\varphi )$ denotes the beam after atmospheric turbulence.

In demultiplexing, the interference of external factors destroys the orthogonality between the vortex beams and causes mode crosstalk; the information of its kth path can be expressed by the following equation.

$$\begin{array}{l}{y}_k(t)=\langle Y_{mux}^{Tx},u_k\left(r,\phi ,z\right)\rangle \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}={\displaystyle \sum _{i=1}^n{x}_i(t){\displaystyle \iint u_i^{’}(r,\phi ,z)}{u}_i^{*}(r,\phi ,z)rdrd\phi }\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}+n(t){\displaystyle \iint u_i^{*}(r,\phi ,z)}rdrd\phi \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}=x_k(t){\displaystyle \iint u_i^{’}(r,\phi ,z)}{u}_i^{*}(r,\phi ,z)rdrd\phi \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}+\hspace{0.17em}{\displaystyle \sum _{i=1,i\ne k}^n{x}_i(t){\displaystyle \iint u_i^{’}(r,\phi ,z)}{u}_i^{*}(r,\phi ,z)rdrd\phi }\\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}+n(t){\displaystyle \iint u_i^{*}(r,\phi ,z)}rdrd\phi \end{array}$$

(8)

3. Residual Hybrid Attention Network Model

3.1. Residual Network Model

Continuously increasing the depth of the network is not only easy to overfit but also very easy to obtain a local optimal solution, and the convolution operation will lead to differences in the extracted features between pixels that are originally connected due to the difference in the perceptual field size, resulting in a large reconstruction aberration. To address the above problems, this paper uses the idea of layer-hopping connection to construct a constant mapping of feature maps, which enables the later layers to learn the residuals of the network and reduce the risk of overfitting. Combined with global contextual information, an attention mechanism is added to establish connections between features and increase the wavefront reconstruction ability of the network. The wavefront reconstruction system based on the residual network is shown in Figure 2.

The backbone network is the ResNet50 model, which first transforms the input light intensity map into a feature image by 7 × 7 downsampling convolution, which is followed by a maximum pooling operation to reduce the parameters of the feature map. The hybrid attention module is then embedded into the ResNet model to fully capture the feature information. Dropout is added to prevent overfitting, and finally, the three-dimensional feature map is mapped to the one-dimensional feature map by the fully connected layer, and the final output is the Zernike coefficient corresponding to the wavefront aberration in the input light intensity map.

3.2. Hybrid Attention Structure

To compensate for the limited sensory field of deep ResNet and the lack of cross-channel interaction, this paper adds a spatial attention (SA) mechanism and channel attention (CA) mechanism to the network, whose main architecture is derived from the dynamic Selective Kernel Neural Networks (SKNet) [21]. The information related to the feature mapping is mainly extracted by convolution operation, and the residual hopping structure is added to reasonably allocate computational resources to the model according to the importance of the input features. By integrating the spatial attention module and the channel attention module, a hybrid attention mechanism model was constructed. The structure is shown in Figure 3.

For spots of different sizes in the light intensity image, the scale of the features varies equally. If convolution kernels of the same size are used, the ability to express the feature information of small spots gradually becomes weaker, leading to an increase in reconstruction aberration error. To address this, this paper uses SKNet, a multi-scale feature extraction network, to enable the model to enhance the representation of shallow features, which facilitates the regression of smaller targets in the network and allows the model to focus more on the targets themselves.

SKNet can be divided into three parts: decomposition, fusion and selection. In the decomposition part, convolutional kernels of different sizes are selected for feature mapping operations. Although the multi-branch structure can tap richer semantic information and enhance the expressiveness of the network, it also increases the difficulty of network training. Therefore, a two-way network structure of 3 × 3 and 5 × 5 size is chosen in this paper, which is represented by the following equation.

$$\tilde{U}=f_{3\hspace{0.17em}\times \hspace{0.17em}3}(X)\in R^{h\hspace{0.17em}\times \hspace{0.17em}w\hspace{0.17em}\times \hspace{0.17em}C}$$

(9)

$$\widehat{U}=f_{5\hspace{0.17em}\times \hspace{0.17em}5}(X)\in R^{h\hspace{0.17em}\times \hspace{0.17em}w\hspace{0.17em}\times \hspace{0.17em}C}$$

(10)

where $X\in R^{h\hspace{0.17em}\times \hspace{0.17em}w\hspace{0.17em}\times \hspace{0.17em}C}$ is the input feature map and $f$ represents the convolution operation so that not only the feature representation of the high and low-frequency parts of the light intensity image can be obtained through the multi-branch structure, but also, too many network branches can be avoided to cause training difficulties in convergence. The 5 × 5 convolution uses the null convolution operation.

In the fusion part, the feature maps are summed by the summation operation control, and then, $s$ is obtained by global average pooling the information carried by each channel of the statistical convolution layer, and finally, feature dimensionality reduction is performed to obtain $z$. The overall process is shown in the following equation.

$$U=\tilde{U}+\widehat{U}$$

(11)

$$s={ℱ}_{gp}(U)=\frac{1}{h\times w}{\displaystyle \sum _{i=1}^h{\displaystyle \sum _{j=1}^{w}U(i,j)}}$$

(12)

$$z={ℱ}_{fc}(s)=\delta (ℬ(W_s))$$

(13)

where ${ℱ}_{gp}$ is the global pooling operation, ${ℱ}_{fc}$ is the fully connected operation, $\delta$ is the Rectified linear unit (Relu), $ℬ$ is the Batch Normalization layer, $W_s\in R^{d\times C}$, $d=\max (C/r,L)$ denotes the feature dimension after full connection, $r$ is the compression factor, and $L$ denotes the minimum value of $d$. The global attention module is used to integrate deep and shallow features to alleviate the problem of losing information of smaller spots caused by the low resolution of deep features.

For the selection part, the softmax function is used to calculate the weight distribution of each channel. The calculation is shown in the following equation.

$$a_c=\frac{e^{A_{c}z}}{e^{A_{c}z}+e^{B_{c}z}},b_c=\frac{e^{B_{c}z}}{e^{A_{c}z}+e^{B_{c}z}}$$

(14)

$$V=a·\tilde{U}+b·\widehat{U},a_c+b_c=1$$

(15)

where $a$ and $b$ are soft attention weight matrices for the two channels, respectively.

The amount of high-frequency information is further reduced due to the presence of pooling layers, resulting in a weaker representation of deep features for small targets. In this paper, we introduce a spatial attention mechanism to weight the features of the target region and find the connection between features in the network so that the feature extraction network selectively focuses on the target region containing important information. The final output matrix is calculated as shown below.

$$\tilde{V}=\sigma (f_{5\hspace{0.17em}\times \hspace{0.17em}5}(c[{ℱ}_{avg};{ℱ}_{\max }]))$$

(16)

$$Y=V·\tilde{V}$$

(17)

where $c$ denotes the splicing operation, $\sigma$ is the sigmoid activation function. ${ℱ}_{avg}$ and ${ℱ}_{\max }$ denote the average pooling operation and the maximum pooling operation performed along the channel axis, respectively. Unlike channel attention, spatial attention can distinguish high and low-frequency information, which is complementary to channel attention [22], and feature maps optimized for spatial information can effectively express the feature similarity between pixel points, and spatial attention can be a good way to obtain contextual semantic information.

3.3. Loss Function

To make the reconstruction of Zernike coefficient of wavefront aberration more reasonable, this paper combines the commonly used wavefront evaluation metrics to improve the loss function of the network. The commonly used AO system evaluation metrics are peak to valley (PV) and root mean square (RMS); PV is used to represent the difference between the highest and lowest points of the wavefront, and RMS characterizes the deviation of the measured beam wavefront compared to the ideal wavefront [23], and its formula is shown below.

$$PV=\max \left(\Delta \varphi \left(\rho ,\theta \right)\right)-\min \left(\Delta \varphi \left(\rho ,\theta \right)\right)$$

(18)

$$Y=V·\tilde{V}$$

(19)

where $\Delta \varphi$ is the wavefront aberration and $\overline{\Delta \varphi }$ is its mean value. $\rho$ and $\theta$ are the polar coordinates of the optical pupil surface.

It can be seen from the equation that the PV value does not reflect the wavefront information comprehensively, while the RMS value focuses more on the overall wavefront information, and combining the two can represent the wavefront situation more accurately. Therefore, the loss function of the combined MSE is shown below.

$$MSE=\frac{1}{N}{\displaystyle \sum _{i=0}^N{\left(z_{act}^i-z_{pre}^i\right)}^2}$$

(20)

$$ℒ={\lambda }_{MSE}{ℒ}_{MSE}+{\lambda }_{PV}{ℒ}_{PV}+{\lambda }_{RMS}{ℒ}_{RMS}$$

(21)

where $z_{act}$ and $z_{pre}$ are the actual and estimated Zernike coefficients, respectively, with length $N$, $\lambda$ is the weight of each loss, $ℒ$ is the weighted sum of each loss function, and the reduction in the loss function represents the improvement of the reconstruction accuracy.

4. Deep Learning-Based Wavefront Recovery Simulation Experiments

To verify the effectiveness of the residual attention network proposed in this paper, experimental simulations of vortex beam transmission are carried out to develop experimental simulations of wavefront recovery at different turbulence intensities. The vortex beam undergoes phase distortion after passing through atmospheric turbulence, as shown by Equation (6). The Zernike coefficient is not statistically independent but follows a Kolmogorov distribution, and its magnitude is related to the $D/r_0$ ratio, where $D$ is the system aperture diameter and $r_0$ is the atmospheric coherence length. For systems with reception apertures up to 1 m [24], $D/r_0=2$ can represent weak turbulence, $D/r_0=10$ can represent medium turbulence, and $D/r_0=20$ can represent strong turbulence. The rest of the parameters [25] are set as shown in

Table 1. Parameter of simulation.

Parameter	Value
$\mathrm{Laser} \mathrm{wavelength} \lambda$	1550 nm
$\mathrm{Width} \mathrm{of} \mathrm{the} \mathrm{phase} \mathrm{screen} D$	0.3 m
$\mathrm{Beam} \mathrm{waist} w_0$	3 cm
Topological charge l	3
Radial index p	0
Transmission distance z	1 km
Number of phase screens	10

.

In addition, to make the experiment more reasonable, a range of ratios was set at an interval of 2 to $D/r_0=2~20$. Each ratio generates 5000 sets of random Zernike coefficients and corresponding light intensity maps as training data. The ratio of the training set, validation set and test set is 8:1:1, the size of the training image is adjusted to 224 × 224 pixels, and the output of the network is 36 Zernike coefficients. To make the experimental comparison clearer, the next test will also generate the corresponding wavefront phase from the output Zernike coefficients for comparison and test the network accuracy from the residuals between the predicted phase and the true phase. The turbulence phase screen and light intensity distribution map shown in Figure 4 can be generated from Equation (3) and the parameters in Table 1. In Figure 4a–c, the phase screens of atmospheric turbulence simulated by Zernike coefficients at different turbulence intensities are represented in degrees, and the PV and RMS values in rad are shown in the figure, with larger values representing larger phase screen undulations and worse atmospheric conditions.

The experimental simulation environment uses the Keras deep learning library in Python language, and the offline training epoch is set to 100, and each batch size is set to 50. The adaptive learning rate Adam algorithm is used to set the initial learning rate to 0.001, and the learning rate decreases to 0.0001 when the network accuracy is not increased within 10 batches. The dropout layer is used in the fully connected layer to prevent network overfitting, and the tanh function is used as the network activation function. Through comprehensive evaluation of experiments, the weights of each part of the loss function in this paper are set to ${\lambda }_{MSE}=1$, ${\lambda }_{PV}=0.5$, ${\lambda }_{RMS}=0.5$. The value of the loss function and the size of the accuracy rate in the training process are shown in Figure 5.

In Figure 5, the loss function in Figure 5a decreases as the number of iterations increases, and the accuracy rate in Figure 5b gradually increases, and finally, the overall accuracy rate can reach about 96%. The variation of loss value (val_loss) and accuracy value (val_acc) in the validation set is close to that in the training set, which indicates that the network structure is reasonably designed and there is no overfitting phenomenon, and its accuracy rate can reach about 97%.

To verify the effectiveness and robustness of the residual attention network model, five sets of Zernike coefficients were randomly generated in the experiment at different turbulence intensities, and the corresponding phase screens were generated as shown in Figure 6a. The Zernike coefficients are predicted by the hybrid attention network model proposed in this paper, and the estimated results and their residual phases are shown in Figure 6b,c, and the results of the literature using CNN are shown in Figure 6d. It can be seen that the residuals predicted by the residual attention network model proposed in this paper have the smallest residuals, and the results predicted by the model using only convolutional neural networks are not accurate enough. The PV and RMS values of the residual phase are shown in

Table 2. PV and RMS values of the residual phase at different turbulence intensities.

	$D/r_0=2$		$D/r_0=5$		$D/r_0=10$		$D/r_0=15$		$D/r_0=20$
	ours	CNN	ours	CNN	ours	CNN	ours	CNN	ours	CNN
$\mathrm{PV}/r$	0.094	0.133	0.135	0.231	0.236	0.485	0.196	0.542	0.335	0.631
$\mathrm{RMS}/r$	0.018	0.029	0.051	0.071	0.085	0.178	0.102	0.242	0.122	0.344

. The aberrated phase residuals predicted by the method in this paper have smaller values and can obtain a more realistic phase screen compared to the previous work. Therefore, it can be seen from Figure 6 and Table 2 that the hybrid attention network has a good reconstruction effect at different turbulence intensities, and the recovered phase screen is similar to the actual phase screen. Figure 7 shows the comparison between the predicted Zernike coefficients and the actual coefficients for five different turbulence intensities, and it can be seen that the predicted results of the method in this paper are closer to the actual coefficients.

After reconstructing the wavefront based on the predicted Zernike coefficients, the correction of the turbulent phase can be achieved by loading an inverse phase wavefront to the distorted beam. To verify the effectiveness of the inverse phase of the residual attention network in the OAM multiplexed communication system, the topological charge of the fixed transmission beam is 1, −2, 3, and −5, the signal-to-noise ratio is set to 10, the modulation is quadratic phase-shift keying, and other conditions remain unchanged, and the experimental simulation is carried out under different turbulence intensities, the results of which are shown in Figure 8. It can be seen that the system BER increases as the turbulence intensity becomes stronger. Compared with the CNN network model, the system with the lowest BER after correction using the model in this paper, the system performance is not improved much after using the Gerchberg–Saxton phase recovery algorithm. The system without the phase correction algorithm has the highest BER, which is indicated by none in the figure, and it can be seen that the BER of the uncorrected signal hardly exceeds the Forward Error Correction (FEC) limit.

To verify the robustness of the residual attention network model, experimental simulations were performed at different signal-to-noise ratios and turbulence intensities, and the results obtained are shown in Figure 9. As can be seen from the figure, the BER increases as the turbulence intensity becomes stronger, but better results than the CNN network can be obtained using the model proposed in this paper at either turbulence intensity. At a BER of 10⁻³, this model has a performance gain of about 1 dB compared to the CNN model. As can be seen from the figure, the signal corrected by the residual attention model has better BER performance and can reach the FEC limit faster than the CNN model.

Distance is one of the key factors affecting the performance of optical communication systems, and the BER of the system grows with the increase in transmission distance. In the experiment, the same simulation conditions were set with a signal-to-noise ratio of 10 and no other conditions were varied, and the comparison was performed at different distances and turbulence intensities, and the results obtained are shown in Figure 10. From the figure, it can be seen that the turbulence effect becomes larger and the system performance becomes worse as the distance increases. At different turbulence intensities, the system corrected using the model in this paper has the lowest BER, which indicates the robustness of the model. As seen in the figure, the residual attention model can reach the FEC limit faster at an SNR of 10.

To verify the effectiveness of the hybrid attention structure proposed in this paper, several different models were trained in the experiments, including the base backbone network ResNet50, the network with CA added, the network with SA added, and the network with the hybrid attention structure added, and the accuracy and running time comparison results were obtained, as shown in

Table 3. Comparison of accuracy and calculation time of different models.

Model	Accuracy	Time/ms	Enhancement (%)
ResNet50	0.885	7.34
ResNet50 + SA	0.921	8.58	4.07%
ResNet50 + CA	0.947	8.77	7.01%
ResNet50 + CA + SA	0.973	9.25	9.95%

.

Table 3 shows that the network with the added attention mechanism has the highest accuracy in reconstructing the Zernike coefficients and the hybrid attention structure has a higher accuracy of 97.3% compared to the single attention structure. In terms of the running time, the average value was taken after testing 1000 times, and it can be seen from the table that the computation time of the added attention network is all below 10 ms, which can meet the requirement of real-time wavefront correction, and the running time can continue to decrease with the increase in hardware performance. The running time of the hybrid attention structure is slightly longer than that of ResNet50, but the performance of the residual attention network proposed in this paper is the best after comprehensive accuracy is evaluated.

5. Conclusions

In this paper, we propose an inverse wavefront phase of the AO system combined with a residual attention network to achieve effective wavefront correction and thus reduce the system BER by addressing the problem of vortex beam distortion caused by atmospheric turbulence, which leads to the degradation of FSO communication quality. The trained model establishes an accurate mapping between the first 36th-order Zernike coefficients and the aberrated light intensity distribution. Experimental simulations were performed under different turbulence conditions, and the obtained wavefront residuals PV and RMS values showed an improvement over previous studies, indicating the strong robustness of the network. At a BER of 10⁻³, the residual attention network has a performance gain of about 1 dB compared to the CNN network. In the case of $D/r_0=2$, PV = 0.094 rad, RMS = 0.018 rad; in the case of $D/r_0=10$, PV = 0.236 rad, RMS = 0.085 rad; in the case of $D/r_0=20$, PV = 0.335 rad, RMS = 0.122 rad. The predicted Zernike coefficients are similar to the actual coefficients, and the phases reconstructed by the coefficients are highly similar to the actual phases. The effectiveness of the hybrid attention network in the reconstructed wavefront phase task is then verified, with the highest accuracy with less increase in time complexity. The high accuracy, real-time performance, and flexibility of the residual attention network provide the practical application of deep learning in AO systems.