Next Article in Journal
Mobile Network Coverage Prediction Using Multi-Modal Model Based on Deep Neural Networks and Semantic Segmentation
Previous Article in Journal
Musculoskeletal Disorder (MSD) Health Data Collection, Personalized Management and Exchange Using Fast Healthcare Interoperability Resources (FHIR)
Previous Article in Special Issue
Latent Prototype-Based Clustering: A Novel Exploratory Electroencephalography Analysis Approach
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Detection of Pilots’ Psychological Workload during Turning Phases Using EEG Characteristics

1
School of Mechatronics Engineering, Shenyang Aerospace University, Shenyang 110136, China
2
Key Laboratory of Rapid Development & Manufacturing Technology for Aircraft, Shenyang Aerospace University, Ministry of Education, Shenyang 110136, China
3
Shenyang Aircraft Corporation, Shenyang 110136, China
4
School of Design & Art, Shenyang Aerospace University, Shenyang 110136, China
*
Authors to whom correspondence should be addressed.
Sensors 2024, 24(16), 5176; https://doi.org/10.3390/s24165176 (registering DOI)
Submission received: 26 June 2024 / Revised: 2 August 2024 / Accepted: 5 August 2024 / Published: 10 August 2024
(This article belongs to the Special Issue EEG Signal Processing Techniques and Applications—2nd Edition)

Abstract

:
Pilot behavior is crucial for aviation safety. This study aims to investigate the EEG characteristics of pilots, refine training assessment methodologies, and bolster flight safety measures. The collected EEG signals underwent initial preprocessing. The EEG characteristic analysis was performed during left and right turns, involving the calculation of the energy ratio of beta waves and Shannon entropy. The psychological workload of pilots during different flight phases was quantified as well. Based on the EEG characteristics, the pilots’ psychological workload was classified through the use of a support vector machine (SVM). The study results showed significant changes in the energy ratio of beta waves and Shannon entropy during left and right turns compared to the cruising phase. Additionally, the pilots’ psychological workload was found to have increased during these turning phases. Using support vector machines to detect the pilots’ psychological workload, the classification accuracy for the training set was 98.92%, while for the test set, it was 93.67%. This research holds significant importance in understanding pilots’ psychological workload.

1. Introduction

The “Statistical Bulletin on the Development of the Civil Aviation Industry in 2021” highlights the rapid growth in aircraft utilization across multiple sectors. In 2021, the civil aviation industry achieved notable metrics, including a total transport turnover of 59.928 billion tonne-kilometers, a passenger turnover of 391.387 billion person-kilometers, and 6.28 million completed flight hours for transport airlines. As the number of flights increases, ensuring safety remains a paramount concern in the contemporary airline industry [1].
Safety is of paramount importance in the contemporary airline industry, especially with the increasing number of flights. According to statistics from the Aviation Safety Network, Figure 1 illustrates the likelihood of accidents at different stages of an airplane’s navigation process. Although the cruise phase has a comparatively lower accident probability—around 30 percent compared to other phases—aviation safety hazards during this phase can jeopardize the lives and property of passengers, leading to unpredictable consequences [2].
Among the numerous factors affecting aviation safety, pilot behavior stands out as one of the most significant. As shown in Figure 2, pilots must maintain prolonged concentration during flight missions [3]. Therefore, studying pilots’ behavior and identifying underlying patterns has become an urgent priority to enhance aviation safety and reduce the occurrence of accidents.
Enhancing pilots’ flight skills stands out as the pivotal measure to reduce the probability of flight accidents and consequently improve flight safety [4]. Continuous monitoring of pilots’ operational processes during flight training is imperative to ensure the timely correction of any irregular operations. Standardizing pilots’ operations is essential to mitigate the safety risks associated with pilot error [5].
Traditionally, flight performance assessment has relied solely on the instructor’s subjective experience. This method’s drawback lies in its susceptibility to the instructor’s subjective influence at any given moment, potentially leading to inconsistent assessment standards. To address this, scholars have proposed various assessment methods based on traditional flight training systems that utilize monitoring data to refine pilot training techniques [6,7].
Recently, researchers began assessing pilots’ conditions during flight training using physiological signal data. Li Jie et al. examined the impact of hypoxic training during high-altitude flights on pilots’ electrocardiographic (ECG) activity and concluded that ECG could effectively monitor the pilots’ training status [8]. Additionally, M. Dilli Babu independently gauged the cognitive load of pilots based on ocular parameters in a military aviation context, finding a significant correlation between pilots’ cognitive load and eye parameters [9].
Electroencephalography (EEG) is a technique for recording electrical activity generated by neurons in the brain, detected on the surface of the scalp. Compared to ECG and EOG, EEG signals offer more direct insights into the brain’s activity changes [10]. EEG has been extensively utilized in various studies to explore different aspects of cognitive and psychological states. G. F. Wilson’s research revealed that while both ECG and EEG data can indicate the pilots’ psychological workload, EEG is more reliable as it directly records the brain’s evoked potentials, providing a clearer reflection of brain activity [11]. Bartosz Binias et al. investigated the correlation between brainwave activity and reaction time in short-haul pilots using EEG data [12,13]. Additionally, Peng Zhang et al. examined brain attributes related to pilots’ error awareness during flight missions, employing EEG in their study [14]. Zhendong Lan et al. analyzed EEG characteristics during fatigue driving and classified fatigue states using support vector machines (SVM) [15]. These studies collectively underscore the importance of EEG in understanding cognitive and psychological states, offering valuable contributions to the field of pilot performance and safety.
This study aims to process and analyze EEG data collected from pilots during left and right turns in simulated flight scenarios. The analysis includes examining the changes in EEG signals, variations in EEG power, and the correlations between EEG power and turning maneuvers. Additionally, NASA-TLX is used to assess the pilots’ psychological workload during cruising and turning phases, and support vector machine (SVM) algorithms are employed to classify their psychological workload. The expected outcome is that applying neuroscience research to flight performance will provide a theoretical foundation for flight training methods, ultimately enhancing flight safety.

2. Materials and Methods

2.1. Participants

The study involved ten professionally trained male pilots, each with at least 50 h of flight simulation training. These participants met the following specific criteria to ensure the consistency and reliability of the experiment:
  • No prior EEG experiment participation: participants had no history of participating in EEG experiments, ensuring that their brain activity recordings were not influenced by prior experience.
  • Good physical health: all pilots were in good physical health, crucial for accurate EEG readings and experimental reliability.
  • Age: participants were aged between 20 and 24 years old, with an average age of 22 years (±2).
  • Right-handedness: right-handedness was a requirement, possibly to maintain consistency in motor responses or due to established correlations between handedness and brain activity.
  • Socio-economic status: participants were from a middle-class socio-economic background.
In order to optimize conditions and minimize external factors [16], the following conditions were observed:
  • Timing: the experiment was scheduled for 9:00 a.m. to minimize the impact of drowsiness, ensuring participants were alert and well-rested.
  • Weather and environmental conditions: the weather was clear, with no wind or other natural factors that could affect the flight simulation or participants’ performance.
  • Health precautions: pilots were instructed to maintain proper eating habits, get adequate sleep, and avoid alcohol or medication for three days before the experiment.
Before the experiment began, participants received a comprehensive briefing on the flight process and experimental equipment, including the EEG cap and other devices. Ethical considerations were also addressed: all participants voluntarily agreed to take part in the experiment and signed a “Consent to Participate in the Experiment” form. At the conclusion of the study, participants were compensated according to the terms outlined in the consent form, ensuring transparency and fairness in their involvement.
These measures collectively aimed to create the optimal conditions for studying EEG data during flight simulation, ensuring that the results would be reliable and applicable to enhancing aviation safety through a better understanding of pilot behavior and psychological workload.

2.2. Experimental Equipment

Flight Simulator: The study utilized a KDKJ-FX-172-6D four-seat Cessna professional flight simul ator. The flight simulator features a six-degree-of-freedom kinetic platform, an accurate aerodynamic model, real-time flight sound effects, a full-scale aircraft joystick, and a variety of virtual terrains to ensure that the flight simulation process is as similar as possible to the actual flight process (as depicted in Figure 3).
EEG cap: In this study, the Emotiv EPOC+ EEG cap was used to record EEG data during pilot driving scenarios. The device comprises fourteen data acquisition channels (AF3, F3, F7, FC5, T7, P7, O1, AF4, F4, FC6, F8, T8, P8, O2), as shown in Table 1.
This EEG cap supports wireless transmission, commonly used for recording action EEG and mitigating the impact of EEG caps on pilots (as depicted in Figure 4) [17]. The cap’s ability to transmit data wirelessly reduces movement restrictions on pilots, making it ideal for dynamic and realistic simulation environments.
Psychological Workload Questionnaire: The NASA-TLX (Task Load Index) is a cognitive load assessment scale developed by NASA. It is a widely recognized subjective psychological workload assessment tool. The scale encompasses six dimensions: mental demand, physical demand, temporal demand, performance, effort, and frustration.

2.3. Experimental Steps

The KDKJ-FX-172-6D flight simulator is set in natural plains scenery, where participants conduct their flight experiments using the simulated flight platform. Each participant performs five segments: take-off, level flight, left turn, right turn, and landing (as depicted in Figure 5). The average aircraft flight altitude is 1000 feet, and the average flight speed is 70 knots [18].
For the left and right turns, pilots follow visual flight rules (VFR) without specific speed or altitude requirements, ensuring they maintain at least a 10 s interval between maneuvers to avoid interference. The detailed flight parameters are as follows:
Take-off phase: The aircraft ascends to 400 feet at a speed of 60 knots.
Climb phase: The aircraft continues to ascend to an altitude of 1000 feet.
Cruise phase: The pilot performs left and right turns.
Descent phase: The aircraft descends back to 400 feet.
Landing phase: The aircraft gradually decreases speed and altitude for a smooth landing.
Throughout the flight, the pilots’ EEG data were continuously recorded using an EEG cap, while changes in cockpit instrumentation and flight attitude were recorded using video equipment.
After completing the experiment, the pilots filled out the NASA-TLX subjective questionnaire based on their flight experiences during the cruising and turning phases. The subjective questionnaire assesses six dimensions of psychological workload. For each dimension, the pilots rated their experiences on a scale from 0 to 10. By calculating the weighted average across different dimensions, the level of psychological workload experienced by the pilots during different flight phases was determined.

2.4. Data Processing

The collected pilot EEG data were first preprocessed to ensure accurate analysis. The various functions and operating states of the brain can be determined based on human behavior and cognition, allowing corresponding EEG signals to be identified for analysis. EEG signals are commonly categorized into four main frequency bands: δ wave, θ wave, α wave, and β wave, as shown in Table 2 [19].
To filter the EEG signals and eliminate the impact of high and low-frequency noise, FIR (finite impulse response) filters were employed within the 0.1–40 Hz range. This preprocessing step is crucial for isolating the relevant EEG signals and improving the accuracy of subsequent analyses [20,21].
ICA is a powerful statistical technique used to decompose multichannel EEG signals into independent components. Each component represents a distinct source of neural activity contributing to the overall EEG signal [22]. During the ICA process, components associated with artifacts such as EMG (muscle activity) and EOG (eye movements) can be identified based on their characteristic patterns and frequencies. These components can then be effectively removed or filtered out from the EEG data. By isolating and removing artifacts through ICA, the quality and reliability of the EEG signals are significantly improved [23]. This allows more accurate analysis of brain activity patterns related to cognitive processes or specific tasks performed during the flight simulation. Figure 6 likely depicts the results of ICA, showing how independent components are extracted from the EEG data and how artifacts are separated from genuine brain signals. This visualization aids in validating the effectiveness of artifact removal and ensuring the integrity of the EEG data for subsequent analysis.

3. Results

3.1. Analysis of EEG Maps in the Left Turn Phase and Right Turn Phase

The EEG (electroencephalogram) data of the pilot were collected throughout the simulated aircraft operation process. During this simulation, the EEG signals specifically associated with beta waves were of particular interest during the pilot’s performance of left and right turns. These beta wave data underwent a series of preprocessing steps and characteristic extractions to isolate the relevant signals from the raw EEG data.
To enhance the quality of the EEG signals, particularly by reducing noise, a method called superimposed averaging was employed. This technique improves the signal-to-noise ratio, making the desired beta wave signals more distinguishable from background noise. After this enhancement, an EEG map was generated. This map provides a visual representation of the energy fluctuations in the beta wave signals across each electrode channel, allowing for a clear and comprehensive view of the pilot’s brain activity during the turning maneuvers [24].
During the cruise phase, the electrodes on the pilot’s EEG typically display light colors, indicating low signal energy levels. However, during a left turn, a notable surge in energy occurs at the T7 electrode, resulting in a darkening of its color, indicating increased neural activity levels. This surge is prominently displayed on the left side of the EEG map, showing heightened energy levels observed in the electrode channels. After the pilot completes the left turn maneuver, the energy levels at electrode T7 gradually diminish, resulting in a return to lighter coloration as the aircraft resumes level flight. Similarly, during right turns, there is an increase in energy at electrode T8, resulting in darker coloration at this electrode site. The right half of the EEG map shows elevated energy levels corresponding to this rise. Once the right turn is completed, the energy levels in each electrode gradually decrease again, indicating a return to level flight. These dynamic changes, including the shifts in energy levels and corresponding color changes, are illustrated in Figure 7, providing a clear visual representation of the pilot’s neural activity during different phases of the flight. However, the map does not facilitate quantitative analysis.

3.2. EEG Signal Correlation

During flight tasks, EEG signals exhibit variations across different electrodes. For correlation analysis, the study selected the beta waves of EEG signals recorded during the left and right turning phases. This study utilized Pearson correlation analysis to examine the EEG signals collected from the various electrodes [25,26]. The Pearson correlation coefficient formula used in this study is as follows:
r = i = 1 n ( X i X ¯ ) ( Y i Y ¯ ) i = 1 n ( X i X ¯ ) 2 i = 1 n ( Y i Y ¯ ) 2
where X i and Y i represent the EEG signal values from two different electrodes at corresponding time points, while X ¯ and Y ¯ are the mean values of these signals over the selected time period. The resulting correlation coefficient r quantifies the linear relationship between the EEG signals from the two electrodes during the turning maneuvers. An r value close to 1 indicates a strong positive correlation, while an r value near −1 signifies a strong negative correlation. A value around 0 indicates no linear relationship between the signals of the electrodes.
In left-turn maneuvers, the highest degree of correlation was observed at electrode T7, indicating a strong correlation between the brain regions associated with T7 during left-turn maneuvers, as depicted in Figure 8.
In contrast, during right-turn maneuvers, the highest degree of correlation was observed at electrode T8, indicating strong correlations between the brain regions associated with T8 during these maneuvers. This finding highlights the necessity for further investigation into the relationship between turning directions and EEG signals, as depicted in Figure 9.

3.3. Extraction of EEG Energy Ratio

The study employed wavelet packet decomposition to extract four distinct rhythmic waves from the EEG signal. Within the frequency range of 0–30 Hz, the EEG signals were decomposed into delta waves (0–4 Hz), theta waves (4–8 Hz), alpha waves (8–12 Hz), and beta waves (12–30 Hz). This method decomposed the EEG signals, enabling the separation and analysis of these rhythmic waves. It yielded insights into varying states of consciousness during flight operations [27,28].
The mother wavelet selected for decomposition was db4, with each sub-band having a bandwidth of 2 Hz [29]. The formula is
W j , k = x , ψ j , k = n x [ n ] ψ j , k [ n ]
where x represents the EEG signal. ψ j , k represents the wavelet basis function. W j , k represents the wavelet packet coefficients of the EEG signal at scale j and position k . x , ψ j , k represents the inner product of the EEG signal and the wavelet basis function. Subsequently, the energy of the EEG signal within various frequency bands is computed utilizing the wavelet packet decomposition coefficients. The EEG energy is computed using the following equation:
E j , k = j , k | W j , k | 2
where E j , k represents the energy at the j scale and k position. Subsequently, the total energy for each rhythm is calculated by aggregating the energies of individual wavelets within specific frequency bands.
For each type of rhythmic wave, the ratio of EEG energy to β/(θ + α) wave energy is computed to assess the proportion of EEG energy relative to the total energy. This method enables further analysis of how the distribution of EEG signal energy changes before and after task performance, providing deeper insights into variations across different task states. Figure 10 visually represents changes in EEG signal energy distribution across task phases.
In contrast to the cruising phase, during the turning phase, there is a significant increase in energy in the beta wave rhythm, with relatively lower energy levels observed in the delta, theta, and alpha wave rhythms, respectively.
Figure 11 illustrates the beta wave energy ratio and the β/(θ + α) wave energy of pilots’ EEG data during different flight phases. These metrics are higher during the turning phases and lower during the cruising phase. This indicates that pilots experience a greater cognitive load and increased brain activity during turns compared to cruising.
This finding is consistent with the understanding that turning maneuvers necessitate more intensive focus and decision-making, thereby increasing the psychological workload on pilots. During turns, pilots must constantly monitor and adjust the aircraft’s orientation, speed, and trajectory, which demands heightened attention and cognitive resources. Conversely, cruising typically involves more stable flight conditions, requiring less active control and allowing for reduced cognitive engagement.
Increased beta wave activity during turns reflects the brain’s response to heightened cognitive demands and stress, as beta waves are often associated with active thinking, focus, and problem-solving. Conversely, relatively lower beta activity during cruising suggests a more relaxed state, where the brain is engaged in routine monitoring rather than intensive problem-solving.

3.4. Extraction of EEG Entropy

EEG entropy serves as a metric for quantifying the complexity and randomness of EEG signals. It is commonly utilized in EEG signal analysis to assess the complexity of brain activity. In this study, we computed both Shannon entropy and sample entropy of the EEG signals to elucidate the variations in brain activity across different tasks [30,31,32].
Shannon entropy measures the average uncertainty or information content in the EEG signals, providing an indication of the overall complexity and unpredictability of brain activity. Higher Shannon entropy values suggest more complex and varied brain activity, while lower values indicate more predictable and less complex activity. The calculation formula is as follows:
H ( X ) = i p ( x i ) log 10 ( p ( x i ) )
where H ( X ) represents the Shannon entropy of signal X , and p ( x i ) represents the probability of the signal taking on value x i . We computed the Shannon entropy associated with the pilot’s EEG signals in both cruising and turning phases, as depicted in Figure 12.
Sample entropy is a nonlinear analysis method used to evaluate the complexity and regularity of time-series data. In EEG signal analysis, sample entropy is used to measure the repeatability or regularity of the signal at different time scales. Specifically, sample entropy can help identify patterns or events in EEG signals and assess the signal’s predictability and the system’s stability. The calculation of sample entropy typically involves defining a window size and a matching tolerance threshold to compare signal segments across different time scales. The calculation formula is
S a m p E n ( m , r , N ) = ln A m ( r ) B m ( r )
where m is the embedding dimension, which defines the length of the sequences that are compared. r denotes the tolerance parameter. N is the length of the time series. A m ( r ) and B m ( r ) are counts of similar sequences in the time series: A m ( r ) is the count of sequences of length m + 1 that are similar within tolerance r , and B m ( r ) is the count of sequences of length m that are similar within tolerance r , as depicted in Figure 13.
By separately computing these two types of entropy, the study aimed to obtain a comprehensive understanding of how brain activity complexity varies across different task phases. Compared to the cruising phase, pilots exhibit higher EEG entropy during turning phases. This increased entropy signifies a higher level of cognitive activity and complexity in brain function during turning maneuvers.

3.5. NASA-TLX Psychological Workload Assessment

Pilots’ psychological workload is subjectively assessed using the NASA Task Load Index (NASA-TLX). This tool collects pilots’ ratings of their psychological workload during different flight phases, specifically cruising and left and right turning phases. Pilots assess their workload across multiple dimensions, rating each dimension on a scale from “0” to “10”. A score of “0” indicates minimal workload, whereas a score of “10” indicates a high workload. This subjective assessment helps in understanding the perceived effort and stress experienced by pilots during different flight maneuvers and complements the objective EEG data analysis, providing a comprehensive view of pilots’ cognitive and psychological states [33].
Additionally, pilots must make pairwise comparisons across six dimensions, selecting the dimension they believe is more closely related to psychological workload, as shown in Figure 14. The formula for calculating the overall psychological workload is
F = i = 1 6 M i × P i 15 ,
where F is the total score of the cognitive workload assessment. M i is the score of the subject in different dimensions. P i is the number of times different dimensions were selected in the weight testing table shown in Figure 14.
Subsequently, reliability and validity analyses were conducted on the weight proportions of different dimensions. The reliability ( α = 0.715 ) and validity ( K = 0.734 ) were found to be acceptable, ensuring the robustness of the weightings used in the workload assessment. The final psychological workload scores were 5.02 for the cruising phase and 6.13 for the turning phase. These results indicate that pilots experience a higher psychological workload during the turning phases than during the cruising phase, emphasizing the increased cognitive demands associated with maneuvering the aircraft during turns.

4. Psychological Workload Detection Model

4.1. Characteristic Parameter

A total of 2700 data sets, randomly selected from a pool of 3000 samples (comprising 1500 turn samples and 1500 cruising samples), were used for training. The remaining 300 data sets were reserved for testing. This approach ensures a balanced representation of both turning and cruising phases in the training data while reserving a separate set of data for evaluating the model’s performance.
Considering the variability in EEG data among different pilots, it was necessary to normalize the EEG energy ratio, Shannon entropy, and sample entropy to ensure consistency and comparability across individuals [34]. Normalization was performed using the following formula:
X n = X X min X max X min ,
where X indicates the original characteristic parameter. X n is the normalized value of X . X min is the minimum value of X in the dataset. X max is the maximum value of X in the dataset.
Firstly, a normality test was conducted on the normalized characteristic parameters. The energy ratio of the EEG wave, β/(θ + α) wave energy, Shannon entropy, and sample entropy all satisfied the assumption of normal distribution ( p > 0.05 ) [35].
Subsequently, Pearson correlation analysis was used to calculate the correlation coefficients between Shannon entropy, sample entropy, and the energy ratio at various flight stages. This statistical method quantifies the degree of linear relationship between these variables, providing insights into how changes in entropy metrics correlate with variations in the energy ratio across different flight phases.
Figure 15 and Figure 16 illustrate the correlation between Shannon entropy and sample entropy with the energy ratio of EEG. The beta band energy ratio and Shannon entropy exhibit the highest correlation among EEG characteristic quantities. The correlation coefficients are presented in Table 3.
Figure 17 shows that, during the turning phases, the correlation between the beta wave energy ratio and the Shannon entropy of EEG signals is stronger than the correlation between the β/(θ + α) wave energy and the Shannon entropy of EEG signals. The Pearson correlation coefficient for left turns is 0.736, and for right turns, it is 0.724. During the cruising phase, the correlation coefficient between these two parameters is 0.682.
Figure 17 clearly demonstrates that both left and right turns are associated with a higher beta wave energy ratio, increased Shannon entropy, and greater psychological workload compared to the cruising phase. Furthermore, the trends of these parameters are consistent.

4.2. Detection Model

Our EEG characteristic detection model utilizes support vector machine (SVM). This method employs a radial basis kernel function to determine the flight phase in which the current EEG characteristics are located. This functionality guarantees high classification accuracy [36]. The radial basis kernel function k ( x i , y i ) is represented by the following function relationship:
k ( x i , y i ) = x i T y i ,
where ( x i , y i ) denotes the training samples. The following system of equations elaborates on the SVM process. The concept of a support vector machine involves finding the separating hyperplane that correctly divides the training dataset and has the maximum geometric margin. The separating hyperplane is
λ x + c = 0 ,
where λ and c represent two parameters in separated hyperplanes. Subsequently, secondary optimization was conducted to determine the optimal boundary between the two variables. The calculation formula is as follows:
min λ , c [ 1 2 λ 2 ] ,
Subsequently, the objective function is transformed into an unconstrained Lagrangian function.
L ( λ , c , ω ) = 1 2 λ 2 i = 1 l ω i [ y i ( ( λ x i ) + c ) 1 ] ,
where ω i represents the Lagrange multiplier. Based on the Karush–Kuhn–Tucker (KKT) conditions, we can derive the solution for the Lagrangian function.
{ L ( λ , c , ω ) λ = 0 L ( λ , c , ω ) c = 0 ,
The primal problem can be transformed into its dual form using Lagrange multipliers, resulting in
max L ( ω ) = i = 1 l ω i 1 2 i = 1 l j = 1 l ω i ω j y i y j ( x i x j ) ,
where ω i and ω j represent the Lagrange multiplier. x i and x j are samples, and y i and y j are the corresponding labels.
The final discriminant function is as follows:
f ( x ) = sgn ( i = 1 k ω i y i ( x i x ) + c )
The NASA-TLX assessment indicates that pilots’ psychological workload is significantly higher during the turning phase compared to the cruising phase. The turning phase is characterized by high workload, whereas the cruising phase is characterized by low workload. The input EEG characteristics comprise the beta wave energy ratio and EEG Shannon entropy, which form the combination with the strongest correlation. The classification results of the pilots’ psychological workload in the test set are depicted in Figure 18.
The classification model results are depicted in Figure 18. The final classification accuracy on the training set is 98.92%. The classification accuracy on the test set is 93.67%.

5. Discussion

The study titled “Detection of Pilot’s Psychological Workload During Turning Phases Using EEG Characteristics” has clarified critical aspects of the cognitive workload experienced by pilots across various flight phases. The incorporation of electroencephalography (EEG) data for this purpose has created new opportunities to improve aviation safety and training methodologies. Several key points and implications stem from this research, warranting further discussion.
Interpretation of Results:
The observed increase in the ratio of beta wave energy and Shannon entropy during left and right turns indicates heightened cognitive activity. This aligns with existing literature that suggests beta waves are linked to active concentration and mental effort. The consistency of these findings across all pilots reinforces the robustness of EEG as a tool for assessing cognitive workload. The correlation analysis further supports these observations, demonstrating strong relationships between changes in EEG signals and specific flight maneuvers. This provides a quantitative basis for understanding the mental demands placed on pilots during various flight phases.
The subjective workload assessments using the NASA-TLX questionnaire aligned well with the EEG-based findings, thereby validating the objective measures. This dual approach strengthens the argument for using EEG in combination with traditional assessment tools to achieve a comprehensive understanding of pilot workload. The higher workload scores reported by the pilots during turns are consistent with the EEG data, thereby reinforcing the reliability of these measures in reflecting actual cognitive states.
The high classification accuracy of the SVM model in distinguishing between different workload states demonstrates the potential of machine learning in cognitive monitoring. The slight drop in accuracy from the training set to the test set is expected and underscores the need for further refinement and validation of the model with larger data sets. The SVM’s ability to classify workload states based on EEG data underscores the feasibility of developing real-time monitoring systems. Such systems could provide immediate feedback to pilots and support personnel, thereby enhancing decision-making processes during flights.
This in-depth analysis will not only enhance our understanding of the role of EEG in monitoring cognitive states but also offer a clearer perspective on the practical applications of these insights. Exploring how these systems can provide immediate feedback and support decision-making processes during flights may pave the way for significant advancements in aviation safety and training methodologies. Highlighting these practical applications may bridge the gap between theoretical research and its implementation, ultimately contributing to more effective and responsive aviation practices.

6. Conclusions

The study employed the EPOC + EEG headset to monitor signal variations in different brain regions of pilots during left and right turns. It analyzed EEG characteristics and the pilots’ psychological workload states using support vector machines for classification [37]. The study’s conclusions are summarized as follows:
EEG Maps and Beta Wave Energy: the EEG maps show a significant increase in beta wave energy at the T7 electrode during the left turn phase and at the T8 electrode during the right turn phase. During the cruising phase, the energy levels at various electrode points remain stable and relatively low.
Entropy and EEG Complexity: the results show that during aircraft turns, the entropy of EEG signals is higher compared to the cruising phase, indicating that EEG activity during the turning phase is characterized by increased complexity and irregularity.
NASA-TLX and Psychological Workload: statistical analysis using the NASA-TLX scale shows that pilots experience a higher psychological workload during turning phases compared to the cruising phase. This increased psychological workload may contribute to the complexity and irregularity of EEG signals during turning maneuvers.
Support Vector Machine Classification: By classifying EEG characteristics using a support vector machine, the psychological workload detection accuracy for the test set was 93.67%. These results indicate that this method has high accuracy in distinguishing the EEG characteristics associated with different levels of psychological workload, providing a reliable foundation for further application in the real-time monitoring of pilots’ psychological workload during flight operations.

Future Work

The gender and age of pilots might influence the ratio of EEG energy and Shannon entropy. Additionally, pilots with varying levels of flying experience might experience different levels of psychological workload during cruising and turning phases. Therefore, further collection of EEG data from a diverse group of pilots is necessary to enhance the SVM dataset and improve classification accuracy. This will help gain a deeper understanding of pilots’ flying states and provide a theoretical basis for pilot training, thereby enhancing flight safety. Considering these additional factors will ensure a more comprehensive and accurate assessment of pilots’ cognitive and psychological states, leading to better training protocols and safer flight operations.

Author Contributions

Conceptualization, L.J., L.Y. and H.L.; data curation, L.Y.; formal analysis, H.L., W.H. and N.Z.; funding acquisition, L.J.; methodology, L.J. and L.Y.; project administration, L.J., H.L., W.H. and N.Z.; supervision, N.Z.; validation, L.J. and L.Y.; writing—original draft, L.Y.; writing—review and editing, L.J. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by 2020 Ministry of Education Humanities and Social Science Research Project (20YJCZH227), Liaoning Province Education Department Youth Fund Project (L2021029), Innovative training program for College Students (D202104012053547909) and Liaoning Province Education Department Project (JYT2020059).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Given the ongoing research value of the data, scholars may reach out to the author to obtain access to it.

Conflicts of Interest

Authors Haiwei Li and Wenjie Han were employed by the company Shenyang Aircraft Corporation. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

  1. CAAC. Report of Civil Aviation Airspace Development in China 2021; China Civil Aviation Annual Report; CAAC: Beijing, China, 2022. [Google Scholar]
  2. Aviation Safety Network. Available online: https://aviation-safety.net/database/2022-analysis (accessed on 26 November 2023).
  3. Donmez, K.; Uslu, S. The effect of management practices on aircraft incidents. J. Air Transp. Manag. 2020, 84, 101784. [Google Scholar] [CrossRef]
  4. Li, M.J.; Zhou, X.; Si, Q.M.; Fu, S.; Guan, H.Y. Investigation method of civil helicopter flight accident basedon fishbone diagram. Sci. Technol. Innov. 2020, 17, 34–35. [Google Scholar]
  5. Boyd, D.D.; Scharf, M.; Cross, D. A Comparison of General Aviation Accidents Involving Airline Pilots and Instrument-Rated Private Pilots. J. Safety Res. 2021, 76, 127–134. [Google Scholar] [CrossRef] [PubMed]
  6. Xu, J. The influence of pilot emotional stability on flight safety and countermeasures analysis. J. Shandong Ind. Technol. 2015, 8, 241. [Google Scholar]
  7. Zhang, G.; Yu, Z. Flight Training Safety Monitoring and Evaluation Equipment Design. Radio Eng. 2021, 51, 1012–1016. [Google Scholar]
  8. Li, J.; Li, S.; Jing, C.; Wang, Y.; Xue, X.; Wang, D.J.; Liu, Q.M.; Xiao, Q. Analysis of ECG changes in high-altitude hypoxia training for high-performance fighter pilots. S. China J. Natl. Def. Med. 2019, 33, 39–41+57. [Google Scholar]
  9. Babu, M.D.; Shree, D.V.J.; Prabhakar, G.; Saluja, K.P.; Pashilkar, A.; Biswas, P. Estimating pilots’ cognitive load from ocular parameters through simulation and in-flight studies. J. Eye Move. Res. 2019, 12, 3. [Google Scholar] [CrossRef] [PubMed]
  10. Aggarwal, S.; Chugh, N. Review of Machine Learning Techniques for EEG Based Brain Computer Interface. Arch. Comput. Methods Eng. 2022, 29, 3001–3020. [Google Scholar] [CrossRef]
  11. Wilson, G.F.; Fullenkamp, P.; Davis, I. Evoked potential, cardiac, blink, and respiration measures of pilot workload in air-to-ground missions. Aviat. Space Environ. Med. 1994, 65, 100–105. [Google Scholar]
  12. Bartosz, B.; Dariusz, M.; Cyran, K.A. A Machine Learning Approach to the Detection of Pilot’s Reaction to Unexpected Events Based on EEG Signals. Comput. Intell. Neurosci. 2018, 2018, 2703513. [Google Scholar]
  13. Binias, B.; Myszor, D.; Binias, S.; Cyran, K.A. Analysis of Relation between Brainwave Activity and Reaction Time of Short-Haul Pilots Based on EEG Data. Sensors 2023, 23, 6470. [Google Scholar] [CrossRef] [PubMed]
  14. Zhang, P.; Yan, J.; Liu, Z.; Zhou, Q. Brain signatures of error awareness during cognitive tasks for humans in the flight environment. Front. Neurosci. 2022, 16, 1007258. [Google Scholar] [CrossRef] [PubMed]
  15. Lan, Z.; Zhao, J.; Liu, P.; Zhang, C.; Lyu, N.; Guo, L. Driving fatigue detection based on fusion of EEG and vehicle motion information. Biomed. Signal Process. Control. 2024, 92, 106031. [Google Scholar] [CrossRef]
  16. Zhang, S.; Zhang, Y.; Sun, Y.; Thakor, N.; Bezerianos, A. Graph theoretical analysis of EEG functional network during multiworkload flight simulation experiment in virtual reality environment. In Proceedings of the 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Jeju, Republic of Korea, 11–15 July 2017; pp. 3957–3960. [Google Scholar]
  17. Chen, K.; Zhang, S.L.; Ding, F. Brain-computer interface research based on Emotiv. Inf. Technol. Netw. Secur. 2018, 37, 116–118. [Google Scholar]
  18. Zhang, N.N.; Fan, S.; Wang, D.D.; Zou, Y.Y.; Ren, H. Research on the cockpit display interface of navigable aircraft based on mental load. Eng. Des. J. 2019, 26, 409–415. [Google Scholar]
  19. Gurudath, N.; Riley, H.B. Drowsy Driving Detection by EEG Analysis Using Wavelet Transform and K-means Clustering. Procedia Comput. Sci. 2014, 34, 400–409. [Google Scholar] [CrossRef]
  20. Jerzy, B.; Paweł, P. Fractional Band-Pass Filters: Design, Implementation and Application to EEG Signal Processing. J. Circuits Syst. Comput. 2017, 26, 1750170:1–1750170:21. [Google Scholar]
  21. Irene, W.; Stefan, D.; Klaus-Robert, M.; Michael, T. On the influence of high-pass filtering on ICA-based artifact reduction in EEG-ERP. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. 2015, 2015, 4101–4105. [Google Scholar]
  22. Yang, C.J.; Nomena; Li, Y.Y.; Zhang, R. Denoising Method of Epilepsy Brain Telecom Number Based on ICA and Wavelet Thresholds. J. Northwest Univ. (Nat. Sci. Ed.). 2016, 46, 795–800. [Google Scholar]
  23. Wang, C.F.; Sun, Y. Study on Eye Electro artifact Removal by Improving Independent Component Algorithm. Comput. Eng. Appl. 2018, 54, 167–173. [Google Scholar]
  24. Yang, Y.; Feng, Y.; Easa, S.M.; Zheng, X. Evaluation of Mental Load of Drivers in Long Highway Tunnel Based on Electroencephalograph. Front. Psychol. 2021, 12, 646406. [Google Scholar] [CrossRef] [PubMed]
  25. Silva, P.; Duarte, R.; Esteves, P.; Travassos, B.; Vilar, L. Application of entropy measures to analysis of performance in team sports. Int. J. Perform. Anal. Sport. 2016, 16, 753–768. [Google Scholar] [CrossRef]
  26. Fu, R.; Du, Y.; Wang, S.; Wen, G.; Chen, J.; Li, Y. Dynamical Differential Covariance Based Brain Network for Motor Intent Recognition. IEEE Sens. J. 2024, 24, 6515–6522. [Google Scholar] [CrossRef]
  27. Wu, B. Analysis of Exercise Intention Based on EEG Rhythm Feature Optimization and Synchronicity; Zhejiang University: Zhejiang, China, 2011. [Google Scholar]
  28. Yang, Y.; Du, Z.; Jiao, F.; Pan, F. Analysis of EEG characteristics of drivers and driving safety in undersea tunnel. Int. J. Environ. Res. Public Health 2021, 18, 9810. [Google Scholar] [CrossRef] [PubMed]
  29. Kar, S.; Bhagat, M.; Routray, A. EEG signal analysis for the assessment and quantification of driver’s fatigue. Transp. Res. F Traffic Psychol. Behav. 2010, 13, 297–306. [Google Scholar] [CrossRef]
  30. Ibrahim, S.; Djemal, R.; Alsuwailem, A. Electroencephalography (EEG) signal processing for epilepsy and autism spectrum disorder diagnosis. Biocybern. Biomed. Eng. 2018, 38, 16–26. [Google Scholar] [CrossRef]
  31. Namazi, H. Information-Based Classification of Electroencephalography (EEG) Signals for Healthy Adolescents and Adolescents with Symptoms of Schizophrenia. Fluct. Noise Lett. 2020, 19, 4. [Google Scholar] [CrossRef]
  32. Devos, H.; Gustafson, K.; Ahmadnezhad, P.; Liao, K.; Mahnken, J.D.; Brooks, W.M.; Burns, J.M. Psychometric Properties of NASA-TLX and Index of Cognitive Activity as Measures of Cognitive Workload in Older Adults. Brain Sci. 2020, 10, 994. [Google Scholar] [CrossRef] [PubMed]
  33. Noyes, J.M.; Bruneau, D.P. A self-analysis of the NASA-TLX workload measure. Ergonomics 2007, 50, 514–519. [Google Scholar] [CrossRef]
  34. Wang, X.; Huang, F.; Cheng, Y. Computational performance optimization of support vector machine based on support vectors. Neurocomputing 2016, 211, 66–71. [Google Scholar] [CrossRef]
  35. Simić, M. Testing for normality with neural networks. Neural Comput. Appl. 2021, 33, 16279–16313. [Google Scholar] [CrossRef]
  36. Hao, Z.; Shaohong, L.; Jinping, S. Unit Model of Binary SVM with DS Output and its Application in Multi-class SVM. In Proceedings of the 2011 Fourth International Symposium on Computational Intelligence and Design, Hangzhou, China, 28–30 October 2011; Volume 1, pp. 101–104. [Google Scholar]
  37. Feng, Z.; Yang, M.; Zhang, W.; Du, Y.; Bai, H. Effect of longitudinal slope of urban underpass tunnels on drivers’ heart rate and speed: A study based on a real vehicle experiment. Tunn. Undergr. Space Technol. 2018, 81, 525–533. [Google Scholar] [CrossRef]
Figure 1. Percentage of fatal accidents.
Figure 1. Percentage of fatal accidents.
Sensors 24 05176 g001
Figure 2. The proportion of factors causing accidents.
Figure 2. The proportion of factors causing accidents.
Sensors 24 05176 g002
Figure 3. Professional flight simulator.
Figure 3. Professional flight simulator.
Sensors 24 05176 g003
Figure 4. Emotiv EPOC+ EEG cap.
Figure 4. Emotiv EPOC+ EEG cap.
Sensors 24 05176 g004
Figure 5. Flight simulation experiment.
Figure 5. Flight simulation experiment.
Sensors 24 05176 g005
Figure 6. Artifact rejection—VEOG.
Figure 6. Artifact rejection—VEOG.
Sensors 24 05176 g006
Figure 7. EEG map before, during and after turns.
Figure 7. EEG map before, during and after turns.
Sensors 24 05176 g007
Figure 8. Spherical correlation graph for left turns.
Figure 8. Spherical correlation graph for left turns.
Sensors 24 05176 g008
Figure 9. Spherical correlation graph for right turns.
Figure 9. Spherical correlation graph for right turns.
Sensors 24 05176 g009
Figure 10. Energy ratios across different task phases.
Figure 10. Energy ratios across different task phases.
Sensors 24 05176 g010
Figure 11. EEG energy characteristics. (a) Beta wave energy ratio; (b) β/(θ + α) wave energy.
Figure 11. EEG energy characteristics. (a) Beta wave energy ratio; (b) β/(θ + α) wave energy.
Sensors 24 05176 g011
Figure 12. EEG Shannon entropy during different task phases.
Figure 12. EEG Shannon entropy during different task phases.
Sensors 24 05176 g012
Figure 13. EEG sample entropy during different task phases.
Figure 13. EEG sample entropy during different task phases.
Sensors 24 05176 g013
Figure 14. NASA-TLX weight test table.
Figure 14. NASA-TLX weight test table.
Sensors 24 05176 g014
Figure 15. Pearson correlation coefficients between Shannon entropy and energy ratio at various flight stages.
Figure 15. Pearson correlation coefficients between Shannon entropy and energy ratio at various flight stages.
Sensors 24 05176 g015
Figure 16. Pearson correlation coefficients between sample entropy and energy ratio at various flight stages.
Figure 16. Pearson correlation coefficients between sample entropy and energy ratio at various flight stages.
Sensors 24 05176 g016
Figure 17. EEG characteristics and psychological workload during different flight maneuvers.
Figure 17. EEG characteristics and psychological workload during different flight maneuvers.
Sensors 24 05176 g017
Figure 18. Classification results of psychological workload.
Figure 18. Classification results of psychological workload.
Sensors 24 05176 g018
Table 1. The corresponding brain regions for EEG electrodes.
Table 1. The corresponding brain regions for EEG electrodes.
ElectrodesLobe
AF3, AF4, F3, F4
F7, F8, FC5, FC6
Frontal
T7, T8Temporal
P7, P8Parietal
O1, O2Occipital
Table 2. The corresponding frequency and physiological meaning of EEG rhythms.
Table 2. The corresponding frequency and physiological meaning of EEG rhythms.
EEG RhythmFrequency (Hz)Amplitude (μV)FunctionMain Location
δ0.5~320~200Sleep, hypoxia, etc.Occipital and parietal areas.
θ4~720~100Burnout, sleep, etc.Frontal and temporal areas.
α8~1310~100Closed eyes, relaxation, etc.Occipital lobe area.
β14~305~20Emotional tension,
thinking activity, etc.
Temporal lobe and
frontal lobe area.
Table 3. Pearson correlation coefficients.
Table 3. Pearson correlation coefficients.
EEG EntropyPhasesδ Waveθ Waveα Waveβ Waveβ/(θ + α)
Shannon entropyTurning left0.2510.4150.3270.7360.662
Cruising0.3190.5710.4820.6820.541
Turning right0.3570.5240.4230.7240.671
Sample entropyTurning left0.2540.4120.3590.5320.571
Cruising0.2850.3250.3670.5140.576
Turning right0.2650.3010.2670.5830.534
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Ji, L.; Yi, L.; Li, H.; Han, W.; Zhang, N. Detection of Pilots’ Psychological Workload during Turning Phases Using EEG Characteristics. Sensors 2024, 24, 5176. https://doi.org/10.3390/s24165176

AMA Style

Ji L, Yi L, Li H, Han W, Zhang N. Detection of Pilots’ Psychological Workload during Turning Phases Using EEG Characteristics. Sensors. 2024; 24(16):5176. https://doi.org/10.3390/s24165176

Chicago/Turabian Style

Ji, Li, Leiye Yi, Haiwei Li, Wenjie Han, and Ningning Zhang. 2024. "Detection of Pilots’ Psychological Workload during Turning Phases Using EEG Characteristics" Sensors 24, no. 16: 5176. https://doi.org/10.3390/s24165176

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop