Feature Extraction in 5G Wireless Systems: A Quantum Cat Swarm and Wavelet-Based Approach

Abstract

This paper represents a new method for the extraction of features from 5G signals using spectrogram and quantum cat swarm optimization (QCSO). The proposed approach uses a discrete wavelet transform (DWT)-based convolutional neural network (W-CNN) to enhance the extracted features and improve the signal classification. The combination of QCSO and W-CNN is designed to enable improved signal recognition and dimension reduction. Our results demonstrate an improvement in the 5G signal feature extraction performance with the use of this novel approach. The QCSO shows improvement in seven out of eight parameters studied when compared to five other state-of-the-art optimization methods.

1. Introduction

In the context of cognitive radio networks, spectrum detection plays an important role in determining the detection and use of available spectrum bands [1]. With recent advances in 5G networks, strong data retention with low-latency communication has become crucial in spectrum sensing applications. Generally, 5G signals can be deployed with various modulation techniques, different bandwidths, and different levels of interference when using traditional spectrum detection methods [2].

In recent years, deep learning models have been used to detect spectrum band levels and identify signal strengths in 5G networks [3]. Convolutional and recurrent deep neural network models have been introduced in various spectrum-sensing applications. The performance of deep learning models in 5G networks was discussed in [4]. These models have shown the best performance in experiments and simulations based on 5G networks [5]. In some real-time applications, the signals can be dynamic and change according to the noise levels; to address this problem, autoencoder models have been used. Moreover, in the work of Li et al., the segregation of the original and noisy signals was performed separately with the help of these models [6].

Spectrum sensing is a key aspect of cognitive radio networks, facilitating the efficient use of the electromagnetic spectrum through the identification and exploitation of vacant frequency bands. In the context of 5G and beyond, where the demand for higher data rates and lower latency is constantly growing, spectrum sensing plays a crucial role in enabling dynamic access to available frequencies [7]. Deep learning techniques have emerged as valuable tools in spectrum sensing due to their ability to enable the automatic identification of complex features in raw signal data. These methods can be adapted to learn from various signal characteristics, making them suitable for applications involving dynamic and heterogeneous 5G networks [8,9]. In terms of spatial and temporal connections of the spectrum signals, it is possible to improve the performance of spectrum sensing technologies by integrating convolutional neural networks (CNNs) and recurrent neural networks (RNNs), as demonstrated by Gao et al [6].

2. Related Works

The huge demand for wireless communication network protocols may be due to the increasing complexity of spectra in dynamic band allocation, especially within unlicensed bands [10]. To overcome these issues, spectrum sensing is adopted, as it enables the allocation of bands in multidimensional states. The signal categorization or modulation mechanism helps identify spectrum-sensing signals. In [10], the deep signal identification network (DSINet) was introduced. This algorithm helps identify the signal strength based on features such as time, frequency, and power. Meanwhile, Tekbyk et al. proposed the use of CNN-based models to detect and identify 5G spectrum signals in wireless communication networks [11]. These models can perform classification without prior communication, operating on two different submodels, such as CASE1 and CASE2. These models can also enable the determination of the signal strength based on convolutional features as well as the removal of unwanted dropout features. Vijay et al. (2023) proposed an RNN-based deep learning model for spectrum detection based on feedback from output states [12]. The efficacy of such models can be assessed using an empirical test using ADALM-PLUTO [13,14].

The application of wavelet transforms in signal identification has been extensively explored in various domains. A computational algorithm was developed by Tse (2006), who utilized wavelet-based transformation to detect power frequency variations, subharmonics, integer harmonics, and interharmonics in power signals. The mentioned study used a continuous wavelet transform (CWT) and a discrete stationary wavelet transform (SWT) to identify and quantify amplifiers of the harmonic frequencies present in the power signal. Ref. [15] focused on employing a wavelet transform in electrocardiogram (ECG) signal analysis for identification purposes. The researchers developed a reliable method to detect R peaks and QRS complexes, using the DWT to extract vital information from the ECG signal. The precise detection of the temporal locations of the R peaks and QRS complex is crucial to the effectiveness of subsequent stages of ECG processing. The mentioned study exhibited a detection precision of up to 99%. Ref. [16] described the use of an optimal-scale wavelet transform to detect weak ultrasonic signals. They explored noise suppression methods using wavelet transforms and introduced a technique to identify the best scale by enhancing the entropy of discrete wavelet transform (DWT) coefficients. This approach was effective in addressing highly noisy ultrasonic signals and proved to be significantly more time-efficient than approaches using traditional wavelet transforms.

In the field of speaker identification, ref. [8] investigated the use of a discrete wavelet packet transform (WPT) in security systems. In their study, they presented a speaker identification system based on the energy of speaker utterances, incorporating signal preprocessing, feature extraction via a wavelet packet transform, and speaker identification through an artificial neural network. The experimental results demonstrated the effectiveness of the proposed speaker identification system. Ref. [17] focused on detecting power quality disturbances using a discrete wavelet transform. The author evaluated various types of disturbances in power quality and proposed the application of a wavelet transform for the rapid and sensitive detection of such disturbances. The above review of the literature highlights the practical applications of wavelet transforms in various fields, including power quality analysis, ECG signal analysis, ultrasonic signal identification, and speaker identification, underscoring the value of wavelet transforms in signal identification and analysis.

Metaheuristic optimization techniques have been extensively used in feature selection in various fields. Houssein et al. (2022) employed a wavelet transform to extract features from epileptic EEG signals and utilized metaheuristic algorithms for feature selection [18]. Singh et al. (2021) focused on the JAYA algorithm, which is a population-based metaheuristic that combines evolutionary algorithms with swarm intelligence. Furthermore, many studies have highlighted the efficacy of bioinspired optimization and feature selection using metaheuristics, emphasizing their effectiveness in diverse applications. Metaheuristic optimization techniques have been explored in the context of the identification of 5G LTE signals, including the hybrid computational positioning and real data-based positioning of small cells in 5G networks and interference management based on metaheuristic algorithms, demonstrating the potential of metaheuristic techniques in optimizing 5G networks [19].

Furthermore, AI and ML have been recognized as catalysts for 5G networks and beyond, offering a range of machine learning algorithms for classification and optimization. In the domain of feature selection for network intrusion detection, metaheuristic algorithms have been used for optimization, and innovative feature classification and selection algorithms have been proposed to enhance the performance of network intrusion detection systems. The above demonstrates the extensive use of metaheuristic optimization techniques for feature selection in various applications, including the identification of 5G LTE signals, as described in

Table 1. Summary of related work.

Related Work	Year	Methodology	Advantages	Disadvantages
Enhanced Spectrum Sensing for 5G Networks using Deep Learning Algorithms	2017	Implementation of deep learning algorithms for improved spectrum sensing in 5G networks	Emphasizes enhancements that are achievable with deep learning	Specific methodologies may vary
5G Signal Detection using Convolutional Neural Networks [20]	2020	Application of convolutional neural networks (CNNs) for identification of different 5G signals	Focuses on a specific deep learning architecture	Limited coverage of alternative methodologies
Spectrum Sensing in 5G with Capsule Networks [21]	2018	Application of capsule networks for spectrum sensing, focusing on capturing hierarchical relationships between features	Improved generalization to varying signal structures	Limited data available for training of capsule networks
Spectrum Sensing in Cognitive Radio Networks using Deep Learning: A Review [22]	2019	Examination of deep learning approaches for spectrum sensing in cognitive radio	Discusses the potential of deep learning in improving spectrum sensing performance	May lack specific focus on 5G and LTE technologies
LTE and 5G Signal Classification Using Recurrent Neural Networks [8]	2020	Utilized a recurrent neural network (RNN) to capture temporal dependencies in signal characteristics, enhancing the classification accuracy	Effective in capturing sequential patterns in signal data	Limited by the sequential nature of signal data, may require longer training times.
Deep Spectrum Sensing: A Survey [3]	2020	Review of various deep learning techniques for spectrum sensing in cognitive radio networks	Provides a comprehensive overview of existing methods	Limited focus on 5G and LTE signals [3]
Hybrid Deep Learning Model for Spectrum Sensing in 5G Networks [23]	2021	Combined CNN and long short-term memory (LSTM) networks to capture both spatial and temporal features in the received signal	Improved accuracy and adaptability to changing signal characteristics	Increased model complexity
Efficient Spectrum Sensing for 5G Using Transfer Learning [24]	2022	Implemented transfer learning from pre-trained models in similar tasks, adapting knowledge to spectrum sensing	Reduced need for extensive labeled data	May suffer from domain shift issues
Deep Learning-Based Spectrum Sensing for 5G and Beyond: Challenges and Opportunities [7]	2021	Investigation of challenges and opportunities in applying deep learning to spectrum sensing for 5G and beyond	Highlights specific issues related to 5G and LTE technologies	Does not provide detailed methodologies
LTE and 5G Signal Detection using Deep Learning: A Comparative Study [1]	2023	Comparative analysis of deep learning techniques for detection of LTE and 5G signals	Offers insights into the performance of different deep learning models	Limited discussion of disadvantages
DeepSweep: Parallel and Scalable Spectrum Sensing via Convolutional Neural Networks [25]	2023	Introduced a shallow CNN-based transceiver design for fast and scalable spectrum sensing	Achieved up to 98% accuracy with reduced training and inference times; outputs in less than 1 ms.	May require integration with existing transceiver designs; performance in diverse environments not fully explored
Deep Neural Networks for Spectrum Sensing: A Review [26]	2023	Employed a convolutional neural network (CNN) for real-time spectrum sensing, extracting hierarchical features from received signals [9].	Achieved high accuracy in signal identification, robustness to noise	Computationally intensive
RL-Based Hyperparameter Selection for Spectrum Sensing With CNNs [27]	2024	Developed a reinforcement learning approach using Q-learning for the optimization of CNN architectures in spectrum sensing tasks	Systematic hyperparameter tuning leading to enhanced detection accuracy; proposed dynamic sensing time adjustment	Increased computational complexity due to reinforcement learning; potential challenges in real-time adaptation

. These techniques have yielded promising results in terms of streamlining feature selection processes and improving system performance.

3. Proposed Work

3.1. DWT-Based Signal Decomposition

Wavelet or scaling functions play a direct role in the computation of signal expansion coefficients. They reflect the relationships between the expansion coefficients on a lower and higher scale [16]. Here, Haar scaling and wavelet functions are used to improve performance with respect to 5G LTE signals. The 5G signal is decomposed into

$$\varphi \left(t\right)=\sum _{a=0}^{M-1}h\left(a\right)\sqrt{2}\varphi (2t-a)=\sum _a^{M-1}h\left(a\right){\varphi }_{1,n}\left(t\right)$$

(1)

where M is the number of decomposition coefficients. For the Haar wavelet, $N=2$, the above relation is expanded as follows:

$$\begin{array}{cc}\hfill \varphi \left(t\right)& =h\left(0\right)\sqrt{2}\varphi (2\ast t)+h\left(t\right)\sqrt{2}\varphi (2t-1)\hfill \\ \hfill & =h\left(0\right){\varphi }_{1,0}\left(t\right)+h\left(1\right){\varphi }_{1,1}\left(t\right)\hfill \end{array}$$

(2)

Here, $h\left(0\right)={\displaystyle \frac{1}{\sqrt{2}}}$ and $h\left(1\right)={\displaystyle \frac{1}{\sqrt{2}}}$ are normalized Haar coefficients. Thus, the Haar wavelet is extended to the $jth$-level decomposition, which is expressed as follows:

$$\varphi (2^{j}t-k)=\sum _{b=2k}^{2k+M-1}h(b-2k)\sqrt{2}\varphi (2^{j+1}t-b)$$

(3)

where $b=2k+n$. In the above equation, b is a variable that depends on the level of decomposition [15,16,20]. For simplicity, we assume that $j=0,k=3$, and $M=2$ for $\left(Haar\right)$, so the above equation is modified as follows:

$$\begin{array}{cc}\hfill \varphi (2^{0}t-3)& =\sum _{b=6}^{7}h(b-2k)\sqrt{2}\varphi (2^{1}t-b)\hfill \\ \hfill \varphi (t-3)& =h\left(0\right)\sqrt{2}\varphi (2t-6)+h\left(1\right)\sqrt{2}\varphi (2t-7)\hfill \end{array}$$

(4)

In the above equation, the Haar values are replaced, and the equation becomes

$$\varphi (t-3)=\varphi (2t-6)+\varphi (2t-7)$$

(5)

Figure 1 shows a hierarchy that indicates how the input image is decomposed; it can be convolved with filters of different sizes to improve the classification performance. The proposed wavelet-based convolutional neural network helps to improve the classification results through the application of filtering and downsampling. After feature extraction, we apply the quantum cat swarm optimization algorithm to achieve feature selection from the extracted features.

3.2. Quantum Cat Swarm Optimization

Quantum cat swarm optimization (QCSO) is inspired by the behavior of cats and involves search and tracking modes. Cats spend most of their time in search mode, in which they rest or observe the environment. When they identify a target, they start to track it and then attack. These two modes are used in QCSO, seeking to simulate the behavior of cats. These modes guide the optimization process in such a way that they mimic the different strategies used by cats in real life [28,29]. In search mode, there are four main components: (1) self-position consideration (SPC), (2) seeking a memory pool (SMP), (3) seeking a range of selected dimensions (SRD), and (4) calculating the number of dimensions to change. Here, SMP defines the memory for each cat, indicating the points pursued by a cat. From the selected dimensions, the mutual ration is calculated based on the SMP; then, the number of dimensions required for node selection is determined using SRD. SPC enables us to check whether the selected nodes are correct or whether additional iterations are needed [17].

The flowchart for quantum cat swarm optimization (QCSO) for 5G spectrogram feature extraction outlines a systematic process for selecting the most relevant time-frequency features from a 5G signal spectrogram, which is shown in Algorithm 1. The process begins by converting the raw 5G signal into a spectrogram using the short-time Fourier transform (STFT) to extract the spectral characteristics of the signal X. The extracted spectrogram features are then used to initialize a quantum cat population, where each cat represents a potential feature subset. The optimization process consists of two modes: seeking mode (search mode) and tracing mode (tracking mode). In seeking mode, multiple copies of each cat’s feature selection vector are generated, and their feature dimensions are randomly adjusted within a defined seeking range of selected dimensions (SRD). The best candidate solution is selected based on a fitness function, which ensures that the chosen features enhance classification accuracy. In tracing mode, the velocity and position of each cat are updated based on the best global feature subset found so far, refining the selection process. The algorithm iterates until convergence, ultimately returning the optimal feature subset, which can be used for efficient 5G signal classification, modulation recognition, or anomaly detection. This approach ensures robust feature selection by balancing exploration and exploitation, leveraging quantum principles for improved optimization performance.

Algorithm 1 Quantum Cat Swarm Optimization (QCSO) for 5G Spectrogram Feature Extraction

1: Step 1: Convert 5G signal to spectrogram   2: $S\leftarrow$ Read 5G signal   3: $Spectrogram\leftarrow STFT\left(S\right)$   4: Step 2: Extract features from the spectrogram   5: $F\leftarrow$ Extract features from $Spectrogram$   6: Step 3: Initialize Quantum Cats   7: $Q_{population}\leftarrow$ Initialize Q-bit individuals representing feature selection   8: Step 4: Optimize using QCSO   9: while not converged do 10:       for each cat i in population do 11:             if cat is in seeking mode (search mode) then 12:                  Generate j copies in existing location nodes, where $j=(SMP-1)$ 13:                  for each copy do 14:                        Select CDC dimensions randomly 15:                        Modify selected dimensions by adding or subtracting a fraction of SRD: 16:                        $X_{jd}\left(new\right)=X_{jd}\left(old\right)+rand\times SRD\times X_{jd}\left(old\right)$ 17:                        Compute new fitness values $F_i$ 18:                        Compute selection probability: 19:                        $P_i={\displaystyle \frac{|F_i-F_b|}{F_{max}-F_{min}}}$ 20:                  end for 21:                  Select best candidate solution 22:             else                   ▹ Tracing mode (tracking mode) 23:                  for each node k and dimension d do 24:                        Update velocity: 25:                        $r_1\leftarrow$ Random value in $[0,1]$ 26:                        $c_1\leftarrow$ Acceleration coefficient 27:                        $V_{k,d}=V_{k,d}+r_1{c}_1(X_{best,d}-X_{k,d})$ 28:                        Update position: 29:                        $X_{k,d}=X_{k,d}+V_{k,d}$ 30:                  end for 31:             end if 32:       end for 33:       Update global best $G_{best}$ 34: end while 35: Step 5: Return best features 36: $Optimal\_Features\leftarrow G_{best}$

From these two processes, we update the cat’s position in the search space. After this, both search mode and tracking mode can be observed. The combined nodes are redistributed across search mode and tracking mode. Finally, we check whether the stopping condition is met. If the condition is fulfilled, the best node features will be selected; otherwise, the loop will be run once again. However, more time will be required to compute the best node features, so we propose a new method called quantum cat swarm optimization. The optimization algorithm combines the principles of quantum computing and the behavior of cats. For example, it relies on qubits, which are also known as quantum bits [30]. In this method, information is represented as qubits, which can exist in multiple states simultaneously using the principle of superposition. Quantum estimation and quantum entanglement are then performed to compute the best node features. Quantum entanglement allows the states of qubits to be correlated; here, the qubit states are directly related to one another. The state of one qubit directly affects the states of other qubits. Cats observe the environment and seek food in the form of targets, so, as they explore the environment, they search for optimal solutions [29]. Cats are known for their ability to acutely focus on targets. Thus, using cat swarm optimization, it is possible to easily find the best nodes in a timely manner, as shown in Figure 2.

The fitness function in QCSO evaluates the quality of the selected feature subsets by balancing classification accuracy and feature subset size. The function is defined as follows:

$$F\left(X\right)={\displaystyle \frac{Accuracy\left(X\right)}{\lambda +\left|X\right|}}$$

(6)

where $Accuracy\left(X\right)$ is the classification accuracy using the selected features X, $\left|X\right|$ is the number of selected features, and $\lambda$ is a regularization parameter that penalizes larger subsets of features to promote feature selection efficiency. The goal is to maximize $F\left(X\right)$, ensuring that the selected features provide high classification performance while minimizing redundancy. This function directly guides the optimization process by influencing the selection and updating of subsets of features throughout the algorithm. To ensure reproducibility, we set $\lambda$ = 0.1 in our experiments, balancing classification accuracy and feature reduction.

This work implements the quantum cat swarm optimization (QCSO) algorithm using a custom quantum simulation approach. In quantum computing, a quantum computer uses qubits to represent information, and these qubits can exist in multiple states simultaneously based on the principles of superposition [17,30]. In this implementation, qubits are simulated using amplitude encoding, where each node in the search space is represented by a quantum state rather than a classical binary representation. The selection process aims to balance between optimal classification accuracy and minimal redundancy of features, ensuring that only the most relevant features are retained. The final number of selected features is influenced by the convergence behavior of the algorithm and the evaluation of the fitness function. This implementation follows a hybrid classical–quantum simulation model, rather than an actual quantum processor execution, to leverage quantum mechanics concepts for feature selection optimization which is shown in

Table 2. Comparison of QCSO with Traditional Metaheuristics for Feature Extraction in 5G Wireless Systems.

Algorithm	Update Mechanism	Key Strengths / Limitations
QCSO	Quantum state update using $\psi =\alpha |0\rangle +\beta |1\rangle$ and rotation gates	Strong global search due to quantum tunneling and superposition; robust in high-dimensional 5G feature spaces
PSO	$v_i(t+1)=w{v}_i\left(t\right)+c_1{r}_1(p_i-x_i)+c_2{r}_2(g-x_i)$	Fast convergence but risk of premature stagnation in complex spaces
CSO	Seeking and tracing mode based on cat behavior	Moderate performance; lacks quantum-scale diversification
GA	Selection, crossover, mutation	Good exploration, but disruptive crossover may hinder exploitation
GWO	$X(t+1)=X_p-A·D$ using alpha, beta, delta wolves	Effective hierarchy but limited adaptability in dynamic 5G scenarios
BAT	Frequency- and loudness-modulated echolocation	Simple and fast, but convergence stalls in high-dimensional problems

.

3.3. Proposed CNN

Convolutional Layers: Given an input vector with a components, $x=(x_0,x_1,x_2,\dots ,x_{a-1})$$\in R^a$, the output size is the same as the input size in terms of the vector component $y=(y_0,y_1,y_2,\dots ,y_{a-1})\in R^a$.

The layout of our network is depicted in Figure 3. The primary network structure is designed similarly to a VGG network. To ensure consistency among the output and input, we exclusively employ $5\times 5$ convolutional kernels and $2\times 2$ padding were used. We substitute the pooling layers with convolutional layers featuring enhanced stride to downsize the feature maps. Using $2\times 2$ padding with stride of 2, the output is halved compared to the input layer size [31]. This method serves as a valuable alternative to maximum pooling without compromising the accuracy. In addition, we integrate projection shortcuts and dense connections to efficiently leverage information from deconstructed images. Through channel-wise concatenation, dense connections directly link each level of a deconstructed image to all subsequent layers. This linkage facilitates the seamless flow of information throughout the network. We next describe the details of two wavelet CNN applications with $1\times 1$. The architecture of the neural network is outlined in detail in the table below, specifying the types of layers, input and output dimensions, activation functions, stride, padding, kernel size, number of kernels, and total parameters for each layer. It starts with an input layer of $256\times 256\times 3$, representing a colored image with dimensions of $256\times 256$ pixels and 3 color channels. The subsequent layers include fully connected layers, wavelet decomposition layers, Conv2D layers with ReLU activation, concatenation layers, and finally dense layers, leading to the SoftMax activated output layer [32]. For instance, the first Conv2D layer (Layer 3) features a $3\times 3$ kernel size, 64 kernels, stride of 1, and the ’same’ padding, resulting in an output size of $112\times 112\times 64$. The parameter counts vary across the layers, with larger convolutional layers contributing significantly to the total number of parameters. This table offers a comprehensive overview of the network architecture, providing information about the characteristics of each layer and its role within the model [33].

4. Results and Discussion

The 5G dataset includes signals, each converted into a spectrogram representation for feature extraction. The spectrograms were generated using short-time Fourier transform (STFT) to capture the time-frequency characteristics of the signals effectively. Regarding the training–validation–test split, we acknowledge the standard 80–10–10 split used in the MathWorks example. However, in our study, we adopted a custom split ratio of 60–20–20, optimized for our specific feature selection and classification task. This decision was based on an extensive evaluation of model generalization performance and computational efficiency. The dataset and processing methodology are clearly documented to ensure reproducibility. In this work, QCSO optimization algorithm is employed to optimize important feature selection in 5G signals. The input parameters for QCSO are as follows: the input population swarm size is 50, the number of iterations is 100, the initial SMP is $0.63$, the initial tracking mode probability is $0.37$, and the rotation angle coefficient is $0.04$. Here, we initialized the values of all parameters in QCSO as follows: SPC: $0.5$, SRD: $0.231$, CDC: 3, maximum velocity limit: $V_{max}=2.0$, acceleration coefficients: $c_1=2.0,c_2=2.0$, these values are varied in each iteration. These values ensure efficient exploration and exploitation in the feature selection process [29]. The rotation angle coefficient controls the step size for updating quantum states in the rotation gate transformation. It ensures that qubits adjust efficiently without excessive oscillation. The tracking mode probability determines the proportion of cats that operate in tracing mode rather than seeking mode, balancing between global exploration and local exploitation for optimal feature selection. For the WCNN to detect the 5G signals based on spectrogram, the input image size is 256 × 256 × 3, representing RGB spectrograms of the signals. Input dataset is split into 60:20:20 ratios, which means $60\%$ of dataset used for training, $20\%$ is used for validation and remaining $20\%$ is used for testing. The learning rate parameter is set to $0.01$ and the WCNN model is trained for 200 epochs with a batch size of 16 using the Adam optimizer. These parameters are optimized to enhance the classification accuracy of 5G signal detection. As shown in

Table 3. Comparison of optimization techniques for 5G signal feature extraction.

Parameter	GA	PSO	GWO	HHO	DE	QCSO
Convergence Speed (Iterations)	1500	800	600	700	1200	450 (Fastest)
Exploration vs. Exploitation Balance	0.6 (Moderate)	0.5 (Exploitation-biased)	0.7 (Balanced)	0.8 (Exploration-biased)	0.75 (Exploration-biased)	0.9 (Best balance)
Feature Selection Accuracy (%)	85.2%	88.6%	90.4%	91.2%	87.5%	95.3%
Computational Complexity (Execution Time in sec)	5.2 s	3.8 s	2.9 s	3.2 s	4.7 s	2.9 s
Robustness (Success Rate Across 50 Runs, %)	80.5%	85.7%	90.1%	91.5%	87.2%	96.8%
Avoidance of Local Minima (%)	65.3%	72.1%	85.6%	88.4%	78.9%	83.2%
Signal Reconstruction Efficiency (SNR in dB)	18.5 dB	21.2 dB	22.8 dB	23.5 dB	20.6 dB	24.1 dB
Energy Efficiency (Joules per Operation)	1.25 J	1.02 J	0.85 J	0.88 J	1.15 J	0.88 J

, QCSO achieves the fastest convergence speed (450 iterations), significantly outperforming traditional methods such as the genetic algorithm (GA) $\left(1500\right)$ and DE $\left(1200\right)$. It also exhibits a well-balanced exploration–exploitation strategy $\left(0.9\right)$, ensuring strong global and local search capabilities. The precision in feature selection $(95.3\%)$ is the highest, demonstrating its effectiveness in identifying relevant features. In terms of computational efficiency, QCSO exhibits the shortest execution time (2.9 s), matching that of gray wolf optimization (GWO), while it is considerably faster than GA (5.2 s) and DE (4.7 s). Furthermore, its robustness (96.8%) ensures consistency across multiple runs, reducing the likelihood of unstable results. QCSO also excels in terms of signal reconstruction efficiency ($24.1$ dB SNR) and energy efficiency (0.88 J per operation), making it the optimal choice for real-time 5G applications. Although Harris Hawks optimization (HHO) and GWO exhibit competitive performance, QCSO consistently outperforms them in several critical parameters. In general, the results confirm that QCSO is the most effective approach, providing the best trade-off between accuracy, speed, robustness, and computational efficiency and optimizing the extraction of 5G signal features.

Among the eight comparative parameters, QCSO shows a consistent improvement in seven. The only exception is the Avoidance of Local Minima, where QCSO achieves 83.2%, slightly lower than HHO’s 88.4%. This minor shortfall is due to the enhanced exploratory mechanisms of HHO driven by stochastic hawk behavior. However, QCSO offers better overall balance (0.9) and faster convergence, which compensates for this gap in practical deployments. Quantum cat swarm optimization (QCSO) combines quantum computing principles with the behavioral modeling of cat swarm optimization (CSO), resulting in enhanced performance across several key metrics in feature extraction tasks for 5G wireless systems. Unlike classical binary or real-valued encodings used in GA or PSO, QCSO represents candidate solutions using quantum bits (qubits), formulated as follows:

$$\psi =\alpha |0\rangle +{\beta |1\rangle \mathrm{with}|\alpha |}^2+{\left|\beta \right|}^2=1$$

This superposition enables simultaneous exploration of multiple states, improving the search capability in high-dimensional feature spaces. QCSO dynamically tunes the exploration-exploitation ratio through quantum rotation gates and adaptive seeking mode behavior. With a balance coefficient of 0.9 (see Table 3), QCSO maintains a better balance compared to PSO (0.5), GA (0.6) and DE (0.75), leading to an effective global and local search. QCSO demonstrates the fastest convergence (450 iterations) and the lowest execution time (2.8 s), indicating efficient navigation through the solution space. This is due to its probabilistic position update mechanism, which avoids the redundant computations typically present in traditional methods. QCSO achieves an avoidance of 83.2% of local minima, higher than GA (65.3%) and PSO (72.1%). This is attributed to the quantum behavior that allows tunneling through local optima barriers, something not possible in conventional metaheuristics. With a feature selection accuracy of 95.3% and a robustness rate of 96.8%, QCSO significantly outperforms other techniques. This is essential to maintain performance under variable 5G channel conditions. QCSO achieves the highest signal reconstruction quality (24.1 dB SNR) and maintains energy efficiency at 0.88 J per operation, showcasing its suitability for power-sensitive 5G environments.

Figure 4 shows the labeled spectrogram for four different classes of 5G signals, distinguishing between different frequency regions. The spectrum is segmented into three distinct zones: new radio (NR), long-term evolution (LTE), and noise. The LTE and NR bands are centrally positioned, surrounded by noise regions. The color scale indicates the intensity of the signal, helping to visualize the signal distribution across the frequency–time domain. The DWT decomposition of the three levels is shown in Figure 5. This figure illustrates the three-level decomposition of 5G LTE signals, where subfigures correspond to different levels of decomposition. The decomposition process enables us to analyze signal variations and extract meaningful frequency components, which are crucial for spectrum detection and classification tasks.

The training accuracy graph in Figure 6 demonstrates the benefits of various neural network architectures, such as ResNet18, ResNet50, ImageNet, GoogleNet, AlexNet, ConvNet, and the proposed wavelet network, in different numbers of iterations. The proposed wavelet network consistently surpassed the other architectures in terms of training accuracy. Starting from an initial accuracy level of 2.94% at 10 iterations, the proposed network steadily improved and achieved 97.78% accuracy at 200 iterations. In contrast, the other architectures showed varying performance, with some exhibiting notable enhancements, while others reached plateaus or showed fluctuations. The proposed wavelet network achieved high training accuracy throughout the iterations, indicating its potential as a robust and efficient neural network architecture for spectrogram images. During the initial iterations (from 0 to 50), the training loss of the proposed wavelet network decreased more rapidly compared to that of most baseline architectures, indicating its learning efficiency in the initial training phases. As training advanced (iterations 50 to 150), the proposed wavelet network consistently recorded lower training losses than the other networks, suggesting its potential to capture patterns and features in the training data. In subsequent iterations (beyond 150), the training loss for all networks, including the proposed wavelet network, appeared to converge, as depicted in Figure 6. This convergence is a typical occurrence as a model adjusts its parameters and the pace of improvement declines. In particular, the proposed wavelet network achieved the lowest training loss in the later iterations, signaling its ability to capture intricate patterns in the data. It should be noted that a lower training loss does not imply superior performance on unseen data (validation or test set). The evaluation of models using different datasets is crucial to determine their generalizability. Furthermore, the consideration of additional metrics such as the validation loss and accuracy would enable a more comprehensive assessment of the model’s performance.

The 5G spectrogram signals were then analyzed, and the resulting spectrograms are depicted in Figure 7 and Figure 8. Figure 9 presents a comparison of various deep learning models, such as ResNet18, ResNet50, ImageNet, GoogleNet, AlexNet, and ConvNet, based on their sensitivity values. Sensitivity, also referred to as recall, reflects a model’s ability to accurately detect positive instances among the total number of actual positives. Models with higher sensitivity values excel in capturing true positive cases. WaveletNet stands out, with a sensitivity value of 0.999, indicating its exceptional performance in correctly identifying positive instances. ResNet50 and ImageNet also demonstrate high sensitivity values of 0.980 and 0.983, respectively, highlighting their effectiveness in capturing true positive cases. These sensitivity results offer insights into the models’ ability to accurately identify relevant features and generate correct predictions. WaveletNet displays particularly strong performance in this regard. Meanwhile, the specificity values for ResNet18, ResNet50, ImageNet, GoogleNet, AlexNet, and ConvNet remain consistently high at 0.999, demonstrating their ability to correctly identify true negative instances. We also note WaveletNet’s superior performance, with a specificity of 1.000, indicating perfect accuracy in detecting true negatives.

This comparison indicates that WaveletNet outperforms the other models in discerning negative instances, highlighting the value of integrating wavelet transforms in image classification tasks. Precision is a metric that reflects the accuracy of the positive predictions generated by a model. Higher precision values indicate fewer false positives. Here, WaveletNet shows the highest precision among the listed models at 0.990, suggesting superior performance in identifying positive instances compared to established architectures such as ResNet18, ResNet50, ImageNet, GoogleNet, AlexNet, and ConvNet, as depicted in Figure 9. Deep learning models also display strong performance, particularly accuracy, after a set number of iterations. ResNet18 and ResNet50 boast high accuracy of 0.916 and 0.928, respectively, while GoogleNet achieves slightly higher accuracy of 0.949. AlexNet exceeds ResNet18 and ResNet50, with accuracy of 0.959. ConvNet achieves accuracy of 0.929. In particular, WaveletNet stands out, with exceptional accuracy of 0.990, indicating superior classification performance. The true positive rate (TPR) and true negative rate (TNR) are illustrated in Figure 10. The TPR reflects the proportion of actual positive instances that are correctly identified by the model. It ranges from 0.894 for ResNet18 to 0.999 for ConvNet and WaveletNet, demonstrating their effectiveness in accurately classifying positive cases. The TNR remains consistently high for all models, ranging from 0.999 to 1.000, suggesting their strong ability to accurately identify negative cases.

Upon comparing the performance metrics across different deep learning architectures, it is found that WaveletNet also outperforms the other models in terms of the false negative rate (FNR), with the lowest value of 0.001, indicating the superior identification of positive instances (Figure 11). ResNet18 and ResNet50 exhibit higher FNR values, implying a higher rate of missed positive instances. Regarding the false positive rate (FPR) and false discovery rate (FDR), ImageNet, GoogleNet, AlexNet, and ConvNet demonstrate similarly low values, with WaveletNet surpassing the lowest FDR of 0.010, indicating the minimal misclassification of negative instances. The comparison emphasizes WaveletNet as a promising model with superior performance in terms of false positives and false negatives, suggesting its potential to result in enhanced accuracy in image classification tasks. Regarding the LR+, Figure 12 again highlights WaveletNet, with the highest value of 2832.110, indicating its successful classification of positive instances. ResNet18 and ResNet50 also show strong LR+ values, while ImageNet, GoogleNet, AlexNet, and ConvNet show lower values. The LR value indicates a model’s ability to distinguish between true negative and false positive predictions. It is found that WaveletNet has the lowest value of 0.001, indicating its excellent performance in minimizing false positives and maximizing true negatives during classification, as shown in Figure 13. In conclusion, the WaveletNet model, which incorporates wavelet transforms, showcases strong potential to yield accurate predictions, offering advantages in scenarios where the minimization of false positives is critical. The consideration of the specific context and the dataset’s characteristics is essential for a comprehensive evaluation of the model’s performance.

5. Conclusions

In this study, we examined the utilization of a discrete wavelet transform (DWT) and quantum cat swarm optimization (QCSO) to extract features from 5G signals. This innovative approach exploits the combined potential of the DWT based CNN and QCSO to enhance the discriminative value of features derived from 5G signals. Furthermore, an innovative wavelet-based convolutional neural network (WCNN) architecture is presented, seeking to maximize the use of the extracted features for enhanced signal classification. The results show the efficacy of the proposed methodology, achieving exceptional performance in categorizing 5G signals. The amalgamation of the DWT and QCSO enabled us to capture the temporal and spectral characteristics of the signals, providing a holistic representation for subsequent classification efforts. The WCNN, tailored with wavelet-based feature extraction insights, further enhanced the accuracy of the classification outcomes. Our research contributes to the expanding knowledge repository in signal processing for 5G communication systems, highlighting the potential to integrate cutting-edge techniques and tools such as the WCNN and QCSO into the feature extraction process. The proposed WCNN framework demonstrates promising results, underscoring the importance of customizing neural network architectures to the distinct attributes of 5G signals.

As we move toward the realm of 5G and beyond, the precise classification and differentiation of signals will become increasingly crucial. Fusion of DWT, QCSO, and WCNN, as described in this study, paves the way for future investigations aimed at optimizing and adapting these methodologies for real-world 5G communication scenarios.