Non-Intrusive Air Traffic Control Speech Quality Assessment with ResNet-BiLSTM

Abstract

In the current field of air traffic control speech, there is a lack of effective objective speech quality evaluation methods. This paper proposes a new network framework based on ResNet–BiLSTM to address this issue. Firstly, the mel-spectrogram of the speech signal is segmented using the sliding window technique. Next, a preceding feature extractor composed of convolutional and pooling layers is employed to extract shallow features from the mel-spectrogram segment. Then, ResNet is utilized to extract spatial features from the shallow features, while BiLSTM is used to extract temporal features, and these features are horizontally concatenated. Finally, based on the concatenated spatiotemporal features, the final speech quality score is computed using fully connected layers. We conduct experiments on the air traffic control speech database and compare the objective scoring results with the subjective scoring results. The experimental results demonstrate that the proposed method has a high correlation with the mean opinion score (MOS) of air traffic control speech.

1. Introduction

Currently, air traffic management in the civil aviation sector in China primarily relies on manual control, with radio communication being used for the instructions exchanged between air traffic controllers and pilots. During the process of voice interaction, factors such as noise, accents, speaking rate, intonation, and timbre can affect the transmission of speech information and auditory perception. However, clear and accurate voice communication between controllers and pilots is a crucial factor in ensuring flight safety. Therefore, the evaluation of the quality of air traffic control speech is of utmost importance.

Speech quality assessment can be broadly categorized into two categories: subjective methods and objective methods. Subjective evaluation, which takes into account the perceptual characteristics of the human auditory system, is widely adopted due to its high credibility and reliability. This method requires professionals to reference the pristine clean version of the speech signal to estimate the quality of the distorted speech signal, typically using a rating scale ranging from 1 to 5 [1], as shown in

Table 1. Speech quality rating scale.

Rating	Speech Quality	Level of Distortion
5	Excellent	Imperceptible
4	Good	Just perceptible, but not annoying
3	Fair	Perceptible and slightly annoying
2	Poor	Annoying, but not objectionable
1	Bad	Very annoying and objectionable

. The average of all ratings is referred to as the mean opinion score (MOS) [2]. During the evaluation process, professionals consider various aspects, including noise, distortion, speech clarity, the naturalness of sound, and alignment with specific application scenarios, which allows for a relatively objective assessment of speech quality.

However, subjective evaluation methods exhibit limitations, encompassing rater subjectivity and variability, as well as the time-consuming, costly, and non-repeatable nature of the evaluation process [3]. To address these challenges, researchers have begun proposing objective evaluation methods. Objective evaluation methods aim to quantify and measure speech quality by using computer algorithms or automation techniques, thus reducing the influence of subjective factors [4]. These methods can be based on acoustic features, waveform analysis, signal processing, and other techniques, using mathematical models and algorithms to assess speech quality. Compared to subjective evaluation methods, objective evaluation methods offer better repeatability and consistency, as well as higher efficiency and lower costs.

Objective evaluation methods can be classified into intrusive and non-intrusive types. Intrusive models also require the pristine clean version of the speech signal as a reference to estimate the quality of distorted speech signals, such as the PESQ model proposed in ITU-T Recommendation P.862 [5,6]. In contrast, non-intrusive models solely rely on the output signal of the transmission system to estimate the MOS, such as the P.563 model introduced by ITU-T [7,8]. Obtaining the pristine clean version of the speech signal can be challenging in practical communication scenarios. Therefore, non-intrusive objective evaluation has become a hot topic in domestic and international research to ensure real-time and convenience. Non-intrusive models can directly assess distorted speech signals, offering greater flexibility to adapt to different application scenarios and reducing the complexity of the evaluation process [9]. This method holds significant importance for a wide range of speech quality assessment tasks.

With the continuous development of technologies like deep learning, researchers are exploring and improving non-intrusive objective evaluation methods to enhance their accuracy and reliability. These methods leverage models such as neural networks to automatically learn speech features and patterns, enabling more precise speech quality assessment. Future research will continue to focus on the advancement of non-intrusive objective evaluation, further driving its application in practical scenarios.

Related Work

Various non-intrusive speech quality assessment methods are available in the literature. A non-intrusive speech quality assessment method under complex environments is proposed in [10,11]. The method [10] employs Bayesian non-negative matrix factorization (BNMF) to calculate the fundamental spectro-temporal matrixes of the target speech, integrating the resulting matrix, as a separate layer, into the deep neural network (DNN) model. Subsequently, a deep neural network is trained to learn the complex mapping between the target source and the mixture signal, reconstructing the magnitude spectrogram of the quasi-clean speech. Finally, the reconstructed speech is regarded as the reference of the modified PESQ to estimate the MOS of the tested speech sample. The method [11] learns an overcomplete dictionary of the clean speech power spectrum through K-singular value decomposition. Then, during the sparse representation stage, it adaptively obtains the stopping residue based on the estimated cross-correlation and noise spectrum, adjusted by an a posteriori SNR-weighted factor. It utilizes orthogonal matching pursuit to reconstruct clean speech spectra from noisy speech. By using the quasi-clean speech as a reference, it estimates the degraded speech’s MOS using the modified PESQ. Both of these methods entail constructing reference signals, leading to high computational costs.

Another approach to non-intrusive speech quality assessment involves extracting intrinsic features from the speech signal and then mapping these features to MOS. Compared to reconstructing clean speech, this method has lower computational costs. Fu et al. [12] proposed an end-to-end non-intrusive speech quality evaluation model based on Bidirectional Long Short-Term Memory (BiLSTM) for predicting speech PESQ scores. However, this model faces difficulties in evaluating enhanced speech with low PESQ scores compared to noisy speech. Chen et al. [13] studied several neural network architectures for evaluating speech conversion quality, including Convolutional Neural Network (CNN), BiLSTM, and CNN–BiLSTM. However, this model is only applied to speech conversion systems and has limited generalization ability. Donald et al. [14] used a pyramid BiLSTM with an attention mechanism to predict MOS, and the proposed model achieved scores close to human judgment. However, the encoder of this model, composed of a stacked pyramid BiLSTM, significantly increases the overall computational complexity, making it less suitable for low-resource speech evaluation tasks. Cauchi et al. [15] combined modulation energy features with a recurrent neural network using Long Short-Term Memory to propose a non-intrusive speech quality evaluation method suitable for evaluating speech enhancement algorithms under various acoustic conditions. Shen et al. [16] proposed a reference-free speech quality evaluation method based on ResNet and BiLSTM. They utilized an attention mechanism [17] to obtain weights and scored based on the BiLSTM output, resulting in scores closer to human ratings. However, this model mostly predicts MOS scores between 1.5 and 4.5, and it cannot effectively predict speech quality with low or high MOS scores.

Most of the aforementioned methods available in the literature show promising results, but, to the best of our knowledge, there is no method available for evaluating the quality of air traffic control speech. This work, inspired by the combined advantages of CNN and RNN and the powerful local feature extraction capabilities of ResNet and BiLSTM, proposes a new air traffic control speech quality assessment method based on ResNet and BiLSTM. This paper considers the importance of air traffic control speech details during the communication process, such as call signs, command types, and command content. Through extensive analysis of experimental data, it retains more air traffic control speech details from three aspects. Firstly, the mel-spectrogram is introduced [18] to capture dense features in the speech, and the mel-spectrogram is segmented for frame-level analysis of speech quality. Secondly, a preceding feature extractor composed of convolutional and pooling layers is employed to compute shallow features suitable for speech quality prediction from mel-spectrogram segments. Lastly, ResNet is utilized to extract spatial features from the shallow features, and BiLSTM is used to extract temporal features from the shallow features [19]. The extracted features from both ResNet and BiLSTM are concatenated and fused, effectively improving the accuracy of air traffic control speech quality evaluation.

In summary, this paper makes the following contributions:

• We created an air traffic control speech database in a real environment, annotated with speech quality scores.
• We proposed a new non-intrusive speech quality evaluation method based on ResNet and BiLSTM for air traffic control speech.
• We investigated the effects of varying signal-to-noise ratios and speech rates in the aerial traffic control speech dataset on the performance of the proposed method.

In the following, Section 2 presents the proposed methodology. Subsequently, Section 3 introduces the database and performance metrics utilized in the experimental setup and the experiments conducted in this study. Finally, Section 4 presents the conclusion and future work.

2. Method

The speech quality evaluation model designed in this paper for air traffic control speech consists of four main components: preprocessing module, shallow feature extraction module, spatiotemporal feature parameter processing module, and score mapping module, as shown in Figure 1. In Module 1, the mel-spectrogram is computed based on the input signal of air traffic control speech, which is then divided into several sub-segments with certain overlapping regions. In Module 2, a feature extractor is designed that computes shallow features suitable for speech quality prediction from mel-spectrogram segments. In Module 3, a parallel network structure of ResNet and BiLSTM is designed to extract spatiotemporal features from shallow features and then splice the features’ output by ResNet and BiLSTM, respectively. In Module 4, a fully connected layer with an attention mechanism is used to estimate the quality of air traffic control speech.

2.1. Preprocessing Module

Air traffic control speech undergoes pre-emphasis, framing, windowing, fast Fourier transform, power spectrum calculation, and filtering through the mel-filterbank to compute the mel-spectrogram [20]. The mel-spectrogram, with MOS scores ranging from 1 to 5, is illustrated in Figure 2.

The model utilizes the sliding window technique [21] to divide the input mel-spectrogram features into several overlapping sub-segments, as shown in Figure 3. Since the mel-filterbank consists of 48 bands, the height of each sub-segment is 48. The width of the sub-segment is set to 15 (equivalent to 15 frames, totaling 40 ms), and the hop size between sub-segments is 4 (equivalent to 4 frames, totaling 40 ms) [22]. By employing this segmentation approach, the model processes air traffic control speech at the frame level, enabling more effective capturing of the temporal features of the speech.

2.2. Shallow Feature Extraction Module

This module employs a preceding feature extractor consisting of convolutional layers and pooling layers to extract features from each mel-spectrogram segment, resulting in a set of local feature representations that better capture short-term variations in the speech signal. The preceding feature extractor consists of 6 convolutional layers, 3 max-pooling layers, and 1 fully connected layer, with specific parameters outlined in

Table 2. Details of the shallow feature extraction network. “-” indicates nonexistent.

Network Layer	Kernel	Stride	Channel	Activation Function
Conv	3	1	16	ReLU
Max-Pooling	2	2	-	-
Conv	3	1	32	ReLU
Max-Pooling	2	2	-	-
Conv	3	1	64	ReLU
Conv	3	1	64	ReLU
Max-Pooling	2	2	-	-
Conv	3	1	64	ReLU
Conv	3	1	64	ReLU
Fully connected	-	-	-	Softmax

. Except for the last convolutional layer, all other convolutional layers utilize width padding. As a result, the input size of $48\times 15$ passes through the sixth convolutional layer to yield an output of $64\times 6\times 1$, where 64 represents the number of channels. Finally, a fully connected layer is used to compress the features into a 384-dimensional vector [23].

2.3. Spatiotemporal Feature Parameter Processing Module

This module utilizes a parallel network of ResNet and BiLSTM for feature extraction and performs feature concatenation and fusion. Such a network structure allows for better capturing of the features and contextual information of the speech signal, providing more accurate and comprehensive high-quality features [24].

2.3.1. ResNet

ResNet is a deep-learning model proposed by He et al. [25] in 2016. It exhibits powerful capabilities in feature extraction and successfully addresses the issue of accuracy saturation or decline with increasing network depth. Figure 4 illustrates the ResNet network, which performs convolution operations in the time dimension to generate feature maps containing temporal sequence information. The ResNet model incorporates batch normalization (BN) layers, establishing correlations among batch samples [26]. As a result, during random batch processing in the network, the model’s output is not uniquely determined for a specific training sample. This characteristic helps, to some extent, in avoiding overfitting issues.

The formula for the ResNet network is shown below [27]:

$$\begin{gathered} y=f\left(x,W\right)+x \\ H\left(x\right)=g\left(y\right) \end{gathered}$$

(1)

where $x$ represents the input from the previous module; $W$ is the weight matrix of the convolutional layer; $f\left(x,W\right)$ denotes the output after two convolutions of $x$; $y$ represents the residual output; g refers to the ReLU activation function; and $H\left(x\right)$ represents the output of $x$ after passing through the ResNet model.

The ResNet model, applied for feature extraction in air traffic control speech, is capable of learning more robust and discriminative feature representations to capture spatial features in speech. Leveraging the advantages of deep learning, the ResNet model demonstrates good generalization performance on different databases compared to other traditional methods [28]. The parameters of each layer in ResNet in this paper are shown in

Table 3. Details of the ResNet network.

Network Layer	Kernel	Stride	Channel	Activation Function
Conv	3	1	32	ReLU
BN	-	-	-	-
Conv	3	1	32	ReLU
Conv	3	1	64	ReLU

.

2.3.2. BiLSTM

BiLSTM is based on LSTM, which is a type of recurrent neural network (RNN). The design goal of LSTM is to address the issues of vanishing and exploding gradients that commonly occur in RNNs when dealing with long sequence information [29]. LSTM introduces gate control mechanisms, including the “forget gate”, “input gate”, and “output gate” [30], which enable long-term memory when processing long sequence information. The structure of LSTM is illustrated in Figure 5.

The computation steps of LSTM are as follows. Firstly, the previous timestep’s state value, $h_{t-1}$, and the current timestep’s input information, $x_t$, are used to calculate the values of three gates and the candidate memory cell, ${\tilde{C}}_t$.

$$\begin{gathered} f_t=\sigma \left(W_f\times \left[h_{t-1},x_t\right]+b_f\right) \\ {i}_t=\sigma \left(W_i\times \left[h_{t-1},x_t\right]+b_i\right) \\ {o}_t=\sigma \left(W_o\times \left[h_{t-1},x_t\right]+b_o\right) \\ {\tilde{C}}_t=tanh\left(W_c\times \left[h_{t-1},x_t\right]+b_c\right) \end{gathered}$$

(2)

where $x_t$ represents the current input; $h_{t-1}$ represents the previous timestep’s state value; ${\tilde{C}}_t$ represents the current timestep’s candidate memory cell; $f$ represents the forget gate; $i$ represents the input gate; $o$ represents the output gate; $tanh$ refers to the hyperbolic tangent activation function; $\sigma$ denotes the sigmoid activation function; $W$ represents the weight matrix; and b represents the bias parameters.

Next, the values of the forget gate, $f_t$; input gate, $i_t$; previous timestep’s memory cell, $C_{t-1}$; and candidate memory cell, ${\tilde{C}}_t$, are combined through element-wise multiplication and addition to update the memory cell, $C_t$.

$$C_t=f_t\times C_{t-1}+i_t\times {\tilde{C}}_t$$

(3)

Finally, the memory cell, $C_t$, is processed through the $tanh$ activation function and multiplied by the output gate, $o_t$, to obtain the current timestep’s state value, $h_t$.

$$h_t=o_t\times tanh\left(C_t\right)$$

(4)

Due to the temporal correlation of noise, jitter, or other disturbances affecting air traffic control speech data [31], there is a continuous impact on the quality of speech data within a certain time range, rather than just instantaneously. In the spectrogram of the speech, this temporal correlation is manifested as persistent patterns or trends of these disturbances over time, rather than a completely random distribution. Merely relying on LSTM to extract forward information features from air traffic control speech is insufficient. Therefore, this paper utilizes a BiLSTM composed of both forward LSTM and backward LSTM to integrate the contextual information of air traffic control speech data. The structure of BiLSTM is illustrated in Figure 6.

The computation process of BiLSTM involves performing forward propagation in the forward layer while simultaneously performing backward computation on the input sequence [32]. The final result is the stacking of outputs from both the forward LSTM and the backward LSTM. The design of BiLSTM in this paper is presented in

Table 4. Details of the BiLSTM network.

Network Layer	Unit
BiLSTM	16
Dropout 50%	-
BiLSTM	32

.

2.4. Score Mapping Module

This module weighs the output of the spatiotemporal feature parameter processing module to compute the final speech quality score. The score mapping module, as shown in Figure 7, takes as input a feature matrix of size dtf × L, where dtf represents the feature dimension after processing for each segment, and L represents the number of segments along the time axis. Through a fully connected layer, attention scores are obtained. These scores are masked and normalized using zero-padding and the softmax function. They are then multiplied with the input matrix, y, through matrix multiplication to obtain a weighted average feature vector, z. Finally, z is passed through another fully connected layer to estimate the overall speech quality [33].

3. Experimental Results

3.1. Experiment Settings

3.1.1. Database

This model aimed to evaluate the quality of air traffic control speech, so it was necessary to construct a real air traffic control speech database. We collected real air traffic control speech from the air traffic control agency’s voice communication system (VCS). The speech was encoded using Pulse Code Modulation (PCM). These speech samples covered a wide range of acoustic conditions, including different genders (male/female), various sources of noise (ground noise, aircraft noise, weather noise, etc.), different speaking rates (slow/medium/fast), and different scenarios (ground, tower, approach, etc.). To create a database for speech quality assessment in the air traffic control domain, which includes the air traffic control speech and their corresponding real MOS values, we employed various strategies. These included voice activity detection (VAD) to extract valid audio segments, automatic speaker recognition, and manual verification. Finally, we annotated the air traffic control speech with the real MOS values. The specific process is outlined as follows.

The method of collecting the air traffic control speech is as follows: based on the existing VCS, an additional bypass VCS is installed at each air traffic control position, and communication speech between pilots and controllers is simultaneously collected through both the bypass VCS and the existing VCS. In this process, the existing VCS is set to normal communication mode, receiving and recording the distorted speech data containing the original noise. The bypass VCS is set to monitoring mode, designed to minimize background noise and interference as much as possible while receiving and recording clean speech data that is theoretically free from disturbances such as noise. Thus, the aforementioned collection of original noisy and clear speech data pairs is achieved [34].

The collected clean–distorted speech data pairs undergo preprocessing, including segmentation and speaker discrimination. The process is as follows:

• Continuous communication speech is segmented into command speech segments using VAD, with each segment containing a single-sentence command from one speaker, while non-active speech is discarded.
• A classification model is employed to classify command speech segments into two categories: controller and pilot speech, with pilot speech data being discarded.

The preprocessing procedure is illustrated in Figure 8.

Following the mentioned steps, we collected 50,000 pairs of air traffic control speech data from real-world environments, each pair containing both the clean speech data and the distorted speech data from the same air traffic control speech. Finally, the distorted speech data were annotated with the MOS to be used for training the subsequent air traffic control speech quality assessment model. Due to the time-consuming and labor-intensive nature of obtaining the MOS through subjective evaluations, we employed the PESQ model proposed by ITU to derive the MOS of the distorted speech data. PESQ calculates the quality score of the distorted speech data by analyzing the feature differences between the clean speech data and the distorted speech data from the same air traffic control speech.

As a result, we obtained an air traffic control speech database comprising 50,000 instances, comprising air traffic control speech with various disturbances including noise, along with the corresponding MOS ratings. The sampling rate of air traffic control speech is 16-kHz, and it can be categorized based on the speaker’s gender (male/female), signal-to-noise ratio (SNR), and speech rate, as shown in

Table 5. Classifications and proportions of air traffic control speech database.

Category	Classification	Proportion
Gender	Male	50%
	Female	50%
SNR	$>$15 dB	10%
	10–15 dB	40%
	5–10 dB	40%
	$<$5 dB	10%
Speech rate	Slow	25%
	Medium	50%
	Fast	25%

.

To verify the quality assessment results and ensure a reliable comparison, 80% of this database is used for training the air traffic control speech quality assessment model, while 20% is used for testing the model.

3.1.2. Experimental Environment

The experiments in this paper were conducted on a 64-bit Windows 10 operating system. The CPU used was an 11th Gen Intel Core i5-11300H, and the deep learning framework employed was PyTorch. During the training process, Adam [35] was used as the network optimizer, with an initial learning rate of 0.001. The batch size was set to 40, and the number of epochs was set to 500. An exponential decay learning rate [36] was utilized, where the learning rate decayed by a factor of 0.2 every 15 epochs.

The objective function proposed in [12] was introduced as the loss function for predicting speech quality. It incorporates frame-level prediction errors to obtain speech-level predictions closer to human scores, as shown in Equation (5).

$$O=\frac{1}{S}{\displaystyle \sum }_{s=1}^S\left[{\left(X_s^{^}-X_s\right)}^2+\frac{1}{T_s}{\displaystyle \sum }_{t=1}^{T_s}{\left(X_s^{^}-x_{s,t}\right)}^2\right]$$

(5)

where $S$ represents the total number of speech samples; and $X_s^{^}$, $X_s$, $T_s$, and $x_{s,t}$ denote the true MOS, the predicted MOS, the total number of frames, and the frame-level prediction for the t-th frame of the s-th speech sample, respectively.

3.1.3. Evaluation Index

This paper adopts the root-mean-square error (RMSE) [37] and Karl Pearson’s Coefficient of Correlation (PCC) [38] as performance metrics to evaluate the model. RMSE is used to measure the magnitude of the error between the predicted and true values. PCC is used to assess the strength of the linear relationship between two variables.

Equation (6) calculates the root-mean-square error between the subjective MOS and predicted MOS. A smaller RMSE indicates the better predictive performance of the model [39].

$$RMSE=\sqrt{{\displaystyle \sum }_{i=1}^N{\left(a_i-b_i\right)}^2/N}$$

(6)

Equation (7) calculates the correlation coefficient between the subjective MOS and predicted MOS, and the closer the absolute value of PCC is to 1, the better the predictive performance of the model is [40].

$$PCC=\frac{{{\displaystyle \sum }}_{i=1}^N\left(a_i-a\right)\left(b_i-b\right)}{\sqrt{{{\displaystyle \sum }}_{i=1}^N{\left(a_i-a\right)}^2{\left(b_i-b\right)}^2}}$$

(7)

where $N$ represents the total number of speech samples in the test set, $a_i$ denotes the subjective MOS of the i-th speech sample, $b_i$ represents the predicted MOS of the i-th speech sample, and $a$ and $b$ represent the arithmetic mean of the subjective MOS and predicted MOS, respectively.

3.2. Contrast Experiment

This study conducted comparative experiments with the NISQA model, the P.563 model, and the Quality-Net model. The P.563 model is the non-intrusive objective speech quality assessment standard established by the ITU, while the NISQA and Quality-Net models are non-intrusive speech quality prediction models proposed in recent years. They are all classic models in the field of non-intrusive speech quality assessment, and many speech quality assessment models have been compared with them. Therefore, this paper also compares with the NISQA model, the P.563 model, and the Quality-Net model.

Figure 9 shows the scatter plots of the subjective and objective evaluations for the four models of the air traffic control speech test set. The vertical axis represents the predicted MOS values, while the horizontal axis represents the subjective MOS values. By comparing the distribution of the scatter plots, we can evaluate the prediction consistency and accuracy of the proposed model compared to the NISQA model, the P.563 model, and the Quality-Net model.

According to Figure 9, it can be observed that the scatter distributions in (a), (b), and (d) are closer to the diagonal line compared to (c), indicating that the predicted results of the proposed model, the NISQA model, and the Quality-Net model are more consistent with human evaluations. The difference between (a), (b), and (d) is not significant. To visually compare the performance of the four methods, we plotted third-order polynomial fitting curves and calculated their PCC and RMSE. The specific results can be found in Figure 10 and

Table 6. Performance comparison of four methods.

Method	PCC	RMSE
ResNet–BiLSTM (Ours)	0.87	0.68
NISQA	0.85	0.95
P.563	0.81	1.33
Quality-Net	0.82	0.89

. These figures and tables provide reliable quantitative indicators for comparing the performance of different models and further support the differences between the proposed model, the NISQA model, and the Quality-Net model.

According to the results in Figure 10, it can be observed that compared to the NISQA model, the P.563 model, and the Quality-Net model, the fitting curve of the proposed model is closer to y = x. This indicates that the proposed model has a more accurate prediction of the subjective MOS.

Additionally, based on the data in Table 6, it can be observed that the proposed model exhibits higher correlation coefficients and lower root-mean-square errors compared to the other models, thus validating the effectiveness of the proposed approach. These results further support the academic contribution of this research and indicate that the proposed model has an advantage in predicting the MOS.

3.3. Impact of SNR

Figure 11 shows the PCC and RMSE of air traffic control speech with varying SNRs. Figure 11 demonstrates that the proposed ResNet–BiLSTM model maintains good performance when evaluating air traffic control speech with different SNRs.

3.4. Impact of Speech Rate

Figure 12 shows the PCC and RMSE of air traffic control speech with various speech rates. Figure 12 demonstrates that the proposed model maintains good performance when the speech rate is medium.

3.5. Discussion

By observing the experimental results, it can be noted that the predicted MOS in this study is generally lower than the subjective MOS. This phenomenon is partly attributed to the influence of using PESQ–MOS as labels for the air traffic control speech database. Since the PESQ method has certain limitations in assessing audio quality, its predictions may deviate from subjective evaluations to some extent. Therefore, training the model with such MOS labels in this study might lead to an underestimation of the subjective MOS. Further exploration and improvement are needed regarding the feasibility of using PESQ–MOS as database labels to enhance the accuracy and reliability of MOS prediction. A better solution would involve annotating the air traffic control speech database with the MOS based on the subjective evaluation methods proposed by ITU.

From Figure 9, we can observe that compared to the NISQA model, the P.563 model, and the Quality-Net model, our proposed approach performs better when predicting real MOS values in the range of 2–3.5 for air traffic control speech. This improvement can be attributed to our optimization of the shallow feature extractor, considering the impact of speaking rate on the mel-frequency spectra of air traffic control speech. Thus, the predictions made by our proposed approach are closer to the real MOS values.

As shown in Figure 12, we found that the performance of the proposed method significantly declines when the speech rate is fast. The underlying reason can be attributed to the fact that, for the same air traffic control speech under identical conditions, as the speech rate increases, the spectrogram of the speech shows dimension reduction. This results in the feature extraction module of the proposed method capturing more redundant information from the spectrogram, leading to deviations in the evaluation of air traffic control speech quality.

4. Conclusions

In this study, a no-reference air traffic control speech quality assessment method was developed by optimizing the details of shallow feature extraction and improving the ResNet–BiLSTM network structure. The experimental results demonstrated that the proposed model accurately estimates the quality of the air traffic control speech and shows a high consistency with subjective evaluations. This provides a feasible solution for air traffic control speech quality assessment.

In the future, we will continue our commitment to enhancing and expanding the practicality and depth of this research. Specifically, we have planned the following initiatives: Firstly, our focus will be on expanding the real air traffic control speech database to encompass a wider range of scenarios and diverse categories of speech samples. This expansion aims to improve the robustness of our model, ensuring its exceptional performance across various contexts. Secondly, we intend to develop a rating standard tailored to air traffic control speech, following the subjective evaluation method proposed by ITU, and apply this standard to rate our speech database. Through subjective evaluations, we will achieve a more accurate quantification of speech quality and validate our model’s performance in real-world scenarios. Additionally, we will continuously refine and optimize the network structure to enhance its capability to accurately predict air traffic control speech quality scores in complex speech environments. This will involve improvements in feature extraction and deep learning architectures to better capture the critical features of speech quality. Lastly, we will conduct an in-depth analysis of the specific factors that influence air traffic control speech quality and convert them into actionable metrics. This will facilitate a deeper understanding of speech quality and further improvements in this domain.