Brainwaves in the Cloud: Cognitive Workload Monitoring Using Deep Gated Neural Network and Industrial Internet of Things

Abstract

Monitoring and classifying cognitive workload in real time is vital for optimizing human–machine interactions and enhancing performance while ensuring safety, particularly in industrial scenarios. Considering this significance, the authors aim to formulate a cognitive workload monitoring system (CWMS) by leveraging the deep gated neural network (DGNN), a hybrid model integrating bi-directional long short-term memory (Bi-LSTM) and gated recurrent unit (GRU) networks. In our experimental setup, each of the four virtual users is equipped with a Raspberry Pi Zero W module to ensure efficient data transmission, thereby enhancing the reliability and efficacy of the monitoring process. This seamless monitoring framework utilizes the constrained application protocol (CoAP) and the Things Board platform to evaluate cognitive workload in real time. The most popular EEG benchmark dataset, the STEW is utilized for workload classification in this study. We employ the short-time Fourier transformation (STFT) to extract frequency bands corresponding to users in both high and low cognitive workload modes. The proposed DGNN models achieve a perfect accuracy of 99.45%, outperforming every previous state-of-the-art model. We meticulously monitored critical parameters, including latency, classification processing time, and cognitive workload levels. This research demonstrates the importance of continuous monitoring for increasing productivity and safety in industries by introducing a novel method of real-time cognitive workload monitoring. The implementation codes for each experiment are documented and made available for reproducibility.

1. Introduction

Cognitive workload is a global issue, alongside the ongoing advancements of the industrial revolution, where new technologies are continually emerging [1]. Industry 4.0 marked a transition from traditional manufacturing methods to digital manufacturing [2], driven by innovations like advanced robot arms [3], industrial Internet of Things (IIoT) [4], artificial intelligence (AI) [5], and cloud computing [6]. These innovations generate a significant amount of data [7] and contribute to a disruptive workplace process, which has become a new source of stress [8]. Industry 5.0, in contrast to its predecessor Industry 4.0, heralds a new era of smooth integration between cutting-edge technology and human–machine interactions [9]. This integration aims to enhance production and efficiency, emphasizing the significance of improving human–machine interactions to boost overall performance [10,11].

In this context, implementing a new cognitive workload monitoring system (CWMS) incorporating deep gated neural network (DGNN) and industrial Internet of Things (IIoT) technologies is essential. This system tracks and classifies cognitive workloads in real time, thereby improving the performance and safety of human–machine interactions. The CWMS holds significance in multiple industrial contexts where human-robot collaboration is pervasive [12,13,14,15]. Cognitive workload, encompassing mental exertion [16] across various functions such as memory [17], decision-making [18], attention [19], perception [20], problem-solving [21], and information processing [22], serves as a foundational element. Continuous monitoring and real-time analysis of workload levels are essential for improving efficiency [23], reducing errors [24], and addressing safety concerns [25], especially in critical industrial environments.

A spectrum of cutting-edge technologies contributes to the assessment of cognitive workload. Electrocardiography (ECG) monitors the heart’s electrical activity [26], providing valuable data on physiological stress levels. Eye-tracking captures gaze patterns, offering insights into visual attention and cognitive processing [27]. Facial expression analysis deciphers real-time emotional states [28], and task performance analysis evaluates metrics related to specific activities [29]. Functional near-infrared spectroscopy (fNIRS), measures changes in blood oxygen levels [30], revealing neural activity associated with cognitive workload. Among all these methods, electroencephalography (EEG) stands out for its effectiveness [31], providing a detailed understanding of cognitive processes with real-time monitoring, non-invasiveness, and portability. Also, these methods leverage machine learning [32], and deep learning [33] algorithms for autonomous identification and classification of cognitive states.

A critical aspect that requires attention is the underexplored intersection of cognitive workload assessment, IIoT, and advanced neural network methodologies for real-time workload monitoring and analysis. There is a lack of comprehensive studies harnessing the combined potential of EEG, IIoT, and refined neural network methods. This creates a gap in developing efficient and accurate cognitive workload monitoring systems that are capable of handling the challenges posed by industrial interactions [1]. In our current study, we aim to bridge this identified gap by focusing on EEG signals for real-time cognitive workload classification and monitoring. Our approach introduces the DGNN model, comprising a dual-layer architecture that incorporates both Bi-LSTM [34] and GRU [35] networks, providing a state-of-the-art framework for cognitive workload classification. The DGNN methodology harnesses the STEW dataset [36], including data from both low and high cognitive workload modes for training and evaluation. The model achieves a remarkable accuracy of 99.45%. Moreover, the article initiates the development of a real-time cognitive workload monitoring system though an experiment that integrates IIoT gateways, the constrained application protocol (CoAP) [37], an AI server, and the Things Board open-source IoT platform for live data monitoring [38]. In this experiment, four virtual users participate, each connected to the AI server, which employs the DGNN model for real-time cognitive workload analysis via IIoT gateways using CoAP. Additionally, the CWMS provides real-time data on parameters such as latency, classification processing time, and cognitive workload levels.

This paper offers the following contributions:

• This study introduces a novel deep gated neural network (DGNN) designed for classifying workloads in real-world scenarios using EEG data.
• The proposed approach achieves superior performance in classifying various workload levels compared to previous studies.
• This study presents a distinctive cognitive workload monitoring system (CWMS) that combines IIoT gateways, CoAP, and an AI server for real-time monitoring and analysis of workload parameters.

The subsequent sections of this article are organized as follows. Section 2 presents a detailed analysis of current studies on the categorization of cognitive workload. Section 3 provides an explanation of the materials and methods used in the study. This includes the preprocessing of EEG data, the design of the DGNN model, and the implementation of the real-time CWMS. In Section 4, the results of both the proposed method and CWMS are presented, along with a thorough analysis. Section 5 provides the conclusion, while Section 6 pertains to the future works.

2. Literature Review

Electroencephalography (EEG) is recognized as a physiological index that is capable of continuously measuring cognitive load [39]. However, EEG signals exhibit characteristics such as weakness, noise, and non-stationarity among subjects, making the identification of robust features a persistent challenge [40]. Traditional methods for cognitive workload classification from EEG data encompass various analytical techniques and feature extraction methods. These include statistical tests, validating differences among features by examining changes in power within specific frequency bands, and time–domain features, such as amplitude, latency, and waveform morphology, providing insights into the temporal characteristics of EEG signals [41,42,43]. Additionally, power spectral density analysis reveals the distribution of power across frequency bands [44], aiding in the inference of workload variations. Event-related potentials (ERPs) reflect the brain’s neural responses to stimuli or events, with the analysis of ERP amplitude and latency elucidating cognitive processing and workload demands [45]. EEG signals are also segmented into different frequency bands, including delta, theta, alpha, beta, and gamma [46], with the activity within these bands providing information about cognitive workload levels, particularly in the gamma band, known for its role in higher cognitive functions [47]. Furthermore, traditional classification methods such as support vector machines (SVM) [48], k-nearest neighbors (kNN) [49], and naive Bayes classifiers [50] have been widely employed for cognitive workload classification based on EEG data.

Significant advancements have been made in understanding cognitive workload using EEG, using machine learning approaches to make valuable contributions towards its assessment. Machine learning algorithms apply features learned from a training dataset to perform tasks such as classification or regression [51]. On the other hand, deep learning is a specific subclass of machine learning that constructs models based on neural networks, such as convolutional networks or long short-term memory networks. Unlike traditional machine learning, deep learning models have the capability to learn complex patterns and representations directly from raw data, eliminating the need for manual feature extraction [52,53]. With automated feature learning, deep learning models can achieve a high recognition accuracy by training directly on raw datasets, allowing for more efficient and accurate cognitive workload classification [53].

In exploring the landscape of cognitive workload detection, several studies have contributed significant insights leveraging machine learning approaches. Momeni et al. (2019) [54] utilized extreme gradient boosting (XGB) algorithms to detect cognitive workload, showing promising results in drone control simulation experiments. Ramírez-Moreno et al. (2021) [55] emphasized the feasibility of short-calibration methods for mental fatigue modeling through biometric signals. Moreno et al. (2021) focused on mental fatigue and human–machine interactions, while Zanetti et al. (2021) [56] addressed job execution support using wearable EEG devices. Cao et al. (2022) [57] introduced a novel framework using hybrid EEG-fNIRS and bivariate functional brain connectivity (FBC) features for workload classification. Additionally, Liu et al. (2023) [58] developed a real-time system for identifying pilots’ cognitive workload levels using a wireless EEG headset, contributing to EEG-based methodologies for real-time assessments in aviation. Despite achieving a lower accuracy rate compared to previous studies, Liu et al.’s research provides valuable insights into cognitive workload assessments in aviation contexts.

Studies using deep learning approaches include Sharma et al. (2021) [59], who developed a cascade one-dimensional convolution neural network (1DCNN) and bidirectional long short-term memory (BLSTM) model for binary and ternary classification of mental workload (MWL). Dolmans et al. (2021) [60] proposed a deep neural network (DNN) that flexibly makes use of multiple modalities, including galvanic skin response, functional near-infrared spectrograms, and eye movements, to classify perceived mental workload (PMWL) accurately on a seven-level workload scale. Afzal et al. (2023) [61] proposed a bi-directional gated network (BDGN) for cognitive workload classification in Industry 5.0 settings, integrating LSTM and GRU and demonstrating promising results in a simulated environment. The reviewed recent studies are summarized in

Table 1. Summary of the performance of previous studies reviewed.

Year	Dataset	Proposed Methodology	Metric	Metric Value	Reference
2019	Lab experiment	Extreme gradient boosting	Accuracy	86%	[54]
2021	Lab experiment	Multiple linear regression	Accuracy	88%	[55]
2022	STEW	Random forest model	Accuracy	83.6%	[56]
2022	Technical University Berlin	Support vector machine	Accuracy	83%	[57]
2023	Lab experiment	K-nearest neighbor	Accuracy	87.57%	[58]
2021	STEW	One-dimensional convolution neural network	Accuracy	95.36% and 96.77%	[59]
2021	Lab experiment	Deep neural network	Accuracy	98%	[60]
2023	STEW	Bi-directional gated network	Accuracy	98%	[61]

.

While existing studies have made strides in cognitive workload detection using EEG and traditional methods, they often face challenges with lower classification accuracies and a lack of real-time monitoring capabilities. These limitations hinder their suitability for advanced industrial settings. Furthermore, there is a notable gap in the literature regarding the development of a unique real-time cognitive workload monitoring system tailored specifically for such environments. This study aims to address these shortcomings by introducing a novel approach that achieves a higher accuracy and enables real-time monitoring using advanced technologies like the Things Board platform and IIoT.

3. Materials and Methods

This section introduce a cognitive workload monitoring system that is designed specifically for industrial applications utilizing EEG signals. The focal point of our approach is the proposed deep gated neural network (DGNN), an innovative recurrent neural network (RNN) architecture. The simultaneous task EEG workload (STEW) dataset, sourced from the IEEE database [62], is selected as the source of raw EEG data. This dataset underwent preprocessing to ensure data integrity and optimal conditions for achieving a high classification accuracy. Furthermore, we performed correlation and frequency domain analyses. Our system’s robustness is augmented by the integration of industrial Internet of Things (IIoT) gateways, the constrained application protocol (CoAP), and the Things Board platform. Figure 1 illustrates the proposed cognitive workload monitoring framework, showcasing the system’s workflow, interconnected components, and data flow that underpin the system’s functionality.

3.1. The STEW Dataset

Wei Lun Lim et al. [62] curated this freely accessible dataset, which comprises raw EEG data collected from 48 subjects engaged in a multitasking workload experiment utilizing the SIMKAP multitasking test at Nanyang Technological University. The brain activity of the subjects during rest was recorded prior to the commencement of the test and is also included in the dataset. The STEW dataset offers insights into brain activity during both rest and task execution states. Each subject contributed 19,200 samples for each state, captured using the Emotiv EPOC device, featuring 14 channels: AF3, F7, F3, FC5, T7, P7, O1, O2, P8, T8, FC6, F4, F8, and AF4. The device operates at a sampling frequency of 128 Hz.

During the experiment, each task-specific session lasted for 3 min per subject. The participants engaged in activities such as comparing the numbers in two windows, crossing out matching numbers, and simultaneously answering accompanying questions. Following each stage, the participants rated their perceived cognitive workload on a scale of 1 to 9. Figure 2 provides an overview of the STEW dataset, detailing the experimental setup, data collection process, and the structure of the dataset.

3.2. Data Preprocessing

EEG signals are inherently noisy due to various artifacts introduced during recording, such as eye movements, muscle activity, and head movements. Removing this noise, therefore, becomes a crucial step in the analysis. We have cleaned the raw data and applied a band-pass filter to retain frequencies within the 0.5 to 80 Hz range. In this analysis, the rest mode is considered as the low mode, and task execution is considered as the high mode. The initial data collection time was 3 min, but the first 15 s and the last 15 s were removed by the STEW data team to mitigate inter-task effects. Ratings for subjects 5, 24, and 42 were unavailable and were provided using an average method. Following segmentation, the EEG data were annotated with subjective workload ratings assigned by the participants, categorized into three levels: low (ratings 1–3), average (ratings 4–6), and high (ratings 7–9). For classification purposes, the EEG data from low- and high-workload subjects were merged into two files, each containing 921,600 rows and 15 columns. The first 14 columns represent Emotiv 14 channels EEG data, while the last column represents workload ratings considered as labels. Figure 3a,b showcases the workload ratings assigned by the participants in both low and high modes.

3.3. EEG Channels Correlation Analysis

In this analysis, the correlation between EEG channels was computed separately for the low and high workload modes. This approach provides understanding into the interrelationship and synchronization of neural activity across different brain regions during different workload conditions [63]. By examining these correlation patterns, we can gain better insight into how cognitive workload affects brain dynamics and connectivity. Figure 4 shows the heat map of the correlation analysis of the channels transmitting low mode data. Strong positive correlations are evident between channels 6 and 7, channels 8 and 9, and channels 11 and 12, indicating robust synchronization during periods of high cognitive demand. Moderately positive correlations are observed among channels 2, 4, 8, 9, 10, 11, 12, and 13, suggesting substantial connectivity and interactions between these channels under similar conditions. Weaker positive correlations are found between channels 1, 2, 3, 7, 10, and 13, highlighting lesser but notable degrees of connectivity among these channels. Conversely, negative correlations are observed between Channels 3, 5, 6, 8, 11, and 12, indicating regions where activity patterns are divergent or inversely related during a low workload.

Figure 5 represents the heat map of the correlation analysis of the channels transmitting high-mode data. Channels 7 and 6 demonstrate a robust synchronization, highlighting a strong interaction between these channels. Additionally, channels 8 and 9 exhibit a very strong correlation, indicating highly similar activity patterns. Moderate to high positive correlations are observed among channels 2, 4, 8, 9, 10, 11, 12, and 13, suggesting significant interconnected activity. Channels 11 and 13 show notable synchronization patterns. Conversely, some channels exhibit weak to moderate positive correlations, while others display negative correlations, indicating varying degrees of interaction and divergence in activity. These findings provide a comprehensive view of how different EEG channels interact under high-workload conditions, shedding light on both synchronized regions and areas with contrasting activity patterns.

3.4. Short-Time Fourier Transformation

We conducted a short-time Fourier transform (STFT) analysis to investigate the time-varying frequency characteristics and power distribution of EEG data. A notable advantage of utilizing STFT is its suitability for analyzing non-stationary EEG signals [64]. Unlike conventional Fourier analysis methods, which assume signal stationarity throughout, STFT breaks down the signal into smaller, overlapping windows. This approach is particularly beneficial for EEG data, as it accommodates the dynamic nature of brain activity, which can vary significantly over time due to shifts in cognitive states.

The equation for the short-time Fourier transform (STFT) is as follows:

$$x\left(\tau ,f\right)={\int }_{-\infty }^{\infty }x(t)w(t-\tau )e^{-i2\pi ft}dt$$

(1)

where $x\left(\tau ,f\right)$ is the STFT of the signal $x(t)$ at time $\tau$ and frequency $f$.

$x(t)$ is the input signal.

$w(t)$ is the window function.

$\tau$ represents time.

$f$ represents time.

Initially, the raw EEG data of subject 1 for both low and high workload modes were acquired, and all 14 channels were selected for subsequent analysis. With a sampling frequency of 128 Hz, the data were processed to capture frequency bands with the frequency ranges mentioned in

Table 2. Captured frequency bands.

Band	Frequency Range
Delta	0.5–4 Hz
Theta	4–8 Hz
Alpha	8–13 Hz
Beta	13–30 Hz
Gamma	30–80 Hz

.

The resulting visualizations depicted the power spectrum distribution across different frequency bands for both low mode and high mode conditions. Specifically, Figure 6a,b illustrate the frequency band visualization of subject 1 in the both low mode and high mode.

3.5. Data Preparation

The inherent structure of most natural data often proves inadequate for optimal model training due to its complexity and variability. To address this, our study represents EEG signals in the form of a matrix, which is considered a best practice for handling such multidimensional data. EEG data underwent preprocessing to remove noise and artifacts, ensuring that the signals were clean and reliable for further analysis. Following this, Z-score normalization was performed to standardize the data, bringing all the values into a common scale with a mean of zero and a standard deviation of one.

For the experiments, an 80/20 train–test split was employed, where 80% of the data were allocated for training the model and 20% were allocated for testing its performance. Additionally, 12.5% of the training data were set aside for validation purposes, allowing for an independent evaluation during the training phase. This resulted in a data split of 70% for training, 20% for testing, and 10% for validation. Considering all 48 participants in the analysis, the dimensions of the input data fed into the model were (921600, 15) for the low workload mode and (921600, 15) for the high workload mode. This extensive dataset ensures that the model has sufficient examples to learn from, ultimately enhancing its ability to generalize and perform well on unseen data.

3.6. Deep Gated Neural Network Model

The deep gated neural network (DGNN) is meticulously designed to achieve precise cognitive workload classification by effectively processing EEG data through a layered architecture. It begins with an input layer, followed by two bidirectional long short-term memory (Bi-LSTM) layers, each equipped with 64 and 32 units, respectively, ensuring comprehensive temporal understanding of the EEG data. Subsequently, the model integrates two gated recurrent unit (GRU) layers, each with 64 and 32 units, enhancing the network’s capacity to capture both short and long-range dependencies within the data. The output layer employs a Softmax activation function to categorize predictions into three workload levels: low, average, and high. Figure 7 represents the architecture of the DGNN, detailing the sequence of layers and the number of neurons in each layer.

The detailed formulation of the deep gated neural network (DGNN) is pivotal for accurately capturing and processing the complex temporal dependencies within EEG data, thereby enabling precise classification of cognitive workload levels.

• First bi-directional LSTM layer:

$$\overrightarrow{h_t}=LSTM(x_t,\overrightarrow{h_{t-1}})$$

(2)

Here,$x_t$ represents the input EEG data at time step t, and $\overrightarrow{h_{t-1}}$ is the hidden state. The forward pass captures the temporal dependencies in the forward direction.

$$\overleftarrow{h_t}=LSTM(x_t,\overleftarrow{h_{t+1}})$$

(3)

In the backward pass, $\overleftarrow{h_{t+1}}$ is the hidden state from the next time step. This captures the temporal dependencies in the reverse direction.

$$h_t=[\overrightarrow{h_t},\overleftarrow{h_t}]$$

(4)

The hidden states from both the forward and backward passes are concatenated to form the final hidden state, $h_t$ for the first Bi-LSTM layer. This provides a comprehensive representation of EEG data by considering information from both directions.

2.

Second bi-directional LSTM layer

$${\overrightarrow{h_t}}^{(2)}=LSTM(\overrightarrow{h_t},{\overrightarrow{h_{t-1}}}^{(2)})$$

(5)

Here, $\overrightarrow{h_t}$ is the output from the first Bi-LSTM layer at time step $t,$ and ${\overrightarrow{h_{t-1}}}^{(2)}$ is the hidden state from the previous time step in the second Bi-LSTM layer’s forward pass.

$${\overleftarrow{h_t}}^{(2)}=LSTM(\overleftarrow{h_t},{\overleftarrow{h_{t+1}}}^{(2)})$$

(6)

Similarly, $\overleftarrow{h_t}$ is the output from the first Bi-LSTM layer, and ${\overleftarrow{h_{t+1}}}^{(2)}$ is the hidden state from the next time step in the second Bi-LSTM layer’s backward pass.

$${h_t}^{(2)}=[{\overrightarrow{h_t}}^{(2)},{\overleftarrow{h_t}}^{(2)}]$$

(7)

The final hidden state ${h_t}^{(2)}$ for the second Bi-LSTM layer is obtained by concatenating the forward and backward hidden states. This ensures that the network has a robust understanding of the temporal patterns in the EEG data from both directions across two layers.

3.

First GRU layer:

$$h_t^{(GRU1)}=GRU(h_t^{(2)},h_{t-1}^{(GRU1)})$$

(8)

Here, $h_t^{(2)}$ is the output from the second Bi-LSTM layer at time step $t,$ and $h_{t-1}^{(GRU1)}$ is the hidden state from the previous time step in first GRU layer. The GRU layer captures dependencies in the data while being computationally efficient and addressing the vanishing gradient problem.

4.

Second GRU layer:

$$h_t^{(GRU2)}=GRU(h_t^{(GRU1)},h_{t-1}^{(GRU2)})$$

(9)

Here, $h_t^{(GRU1)}$ is the output from the first GRU layer at time step $t,$ and $h_{t-1}^{(GRU2)}$ is the hidden state from the previous time step in the second GRU layer. This layer further refines the temporal representations, capturing more complex patterns.

5.

Output layer:

$$y=Softmax (W. h_t^{\left(GRU2\right)}+b)$$

(10)

In the output layer, the final hidden state $h_t^{\left(GRU2\right)}$ from the second GRU layer is passed through a dense layer with a Softmax activation function. Here:

$W:$ weight matrix;

$b:$ bias vector;

$y:$ output probabilities for each class (low, average, high workload).

The Softmax activation function ensures that the model’s output represents a probability distribution, enabling clear classification of cognitive workload levels.

The training process of the model is crafted to ensure not just efficiency but also robustness in its learning. The model is trained using the categorical cross-entropy loss function and optimized using the Adam optimizer. During the training process, the model undergoes 50 epochs, with a batch size of 32, effectively learning and reaching convergence. The Adam optimizer utilizes a default learning rate of 0.001, ensuring stable and efficient training dynamics. Additionally, dropout layers are incorporated to prevent overfitting, and 5-fold cross-validation is utilized to ensure robust model evaluation. Upon completion of training, the performance evaluation of the DGNN model is conducted using a set of metrics. The confusion matrix offers a clear overview by comparing the actual workload levels (low, average, high) with the model’s predictions. In addition, we use fundamental evaluation metrics like accuracy [65], precision [66], recall [67], and F1 score [68] to thoroughly evaluate the model’s ability to accurately classify workload levels.

3.7. Cognitive Workload Monitoring System

The DGNN model is deployed on an Ubuntu server (version 20.04) as part of the cognitive workload monitoring experiment (CWME). The experiment involves four virtual users, represented by laptops, whose EEG data is transmitted wirelessly to Raspberry Pi Zero W modules. These modules, selected for their affordability, compactness, and wireless capabilities, establish connections via Wi-Fi to receive real-time EEG data. Each virtual user sends their EEG data to their designated Raspberry Pi module, which functions as a gateway. Operating on a legacy 32-bit OS installed on a 32 GB memory card, these modules transmit live data to the Ubuntu server. While these modules can serve multiple virtual users simultaneously, in this scenario, each module is dedicated to serving a single virtual user. The DGNN model undertakes critical classification tasks, leveraging its advanced neural network architecture for efficient analysis.

The Ubuntu server takes responsibility for processing cognitive workload data and transmit it to Things Board for live monitoring. As a monitoring platform, Things Board presents various parameters such as latency, DGNN classification time, and workload type. To efficiently manage historical data, integration with a PostgreSQL database is employed due to its support for unstructured information. Figure 8 illustrates this four-layer process within a unified framework, showcasing the seamless flow of data from virtual users to the monitoring interface.

Communication across the layers is facilitated by the constrained application protocol (CoAP). It follows a pattern where data move from virtual users (Layer 1) to Raspberry Pi modules (Layer 2) in a one-to-one manner. Then, the data move from the Raspberry Pi modules to the Ubuntu server with the deployed DGNN model (Layer 3), which operates in a many-to-one fashion. Finally, the data are sent from the Ubuntu server to the Things Board module (Layer 4) in a one-to-one manner.

4. Results and Analysis

This section presents the results obtained from the proposed deep gated neural network (DGNN) and compares them with existing studies. Additionally, it includes the results from real-time monitoring in the cognitive workload monitoring system (CWMS). The Jupyter Lab tool was utilized for analysis due to its powerful interactive computing capabilities, and the proposed DGNN model was designed using the Python programming language.

4.1. The Results Obtained from the DGNN Model

The DGNN model integrates bidirectional long short-term memory (Bi-LSTM) and gated recurrent unit (GRU) architectures to analyze EEG data efficiently. The Bi-LSTM and GRU components within the DGNN framework enable comprehensive analysis of temporal dependencies and intricate patterns within EEG signals. Bi-LSTM excels in capturing long-term dependencies, while GRU enhances the model’s ability to learn from sequential data efficiently. Our findings showcase a remarkable achievement in cognitive workload classification. The DGNN model demonstrates an impressive accuracy rate of 99.45%, as depicted in Figure 9a. The accuracy curve, spanning 50 training epochs, reveals a consistent improvement. In contrast, Figure 9b illustrates the curve of loss values, depicting a continual decrease as the model iteratively learns from the training data. These figures provide a clear visualization of the model’s performance dynamics, underscoring its robustness and efficacy in cognitive workload classification. This suggests promising applications in practical industrial settings. It can enhance safety protocols, optimize task allocation, and improve overall productivity and worker well-being.

The confusion matrix serves as a crucial tool for evaluating the accuracy and reliability of cognitive workload level predictions. In Figure 10, the confusion matrix of our DGNN model showcases significant achievements in workload classification. Specifically, the model accurately classified 93,807 instances as ‘0’ for ‘low workload levels’. Furthermore, it correctly identified 98,229 instances as ‘1’ for ‘medium workload levels’, demonstrating its effectiveness in recognizing moderate cognitive workload levels. Moreover, the model achieved precise classifications in 172,444 instances as ‘2’ for ‘high workload levels’, indicating its proficiency in accurately identifying instances of high cognitive workloads.

These findings underscore the importance of the confusion matrix in revealing how effectively the DGNN model handles different workload levels. By quantifying its classification outcomes, we gain valuable insights into the strengths and areas for improvement of the model, ultimately contributing to the enhancement of cognitive workload assessment techniques.

The classification metrics, as presented in

Table 3. Classification metrics.

Labels	Precision	Recall	F1 Score
Low	1.00	1.00	1.00
Average	0.98	0.98	0.98
High	0.98	0.98	0.98

, provide a detailed evaluation of the model’s ability to accurately classify cognitive workload levels based on the STEW EEG dataset. These metrics includes precision, recall, and F1 score values for each workload category, including low, average, and high. With precision scores of 1.00 for low workloads and 0.98 for the average and high workload categories, the model demonstrates a high accuracy in identifying instances belonging to specific workload levels. Moreover, recall scores of 1.00 for low workloads and 0.98 for the average and high workload categories indicate the model’s ability to capture a high proportion of the actual instances within each workload level. The F1 score values further confirm the model’s balanced performance across all workload categories, with values of 1.00 for low workloads and 0.98 for the average and high workload categories.

In our study, we conducted a comparative analysis of our proposed DGNN model with existing studies in cognitive workload classification, encompassing various methodologies including machine learning and deep learning approaches. This analysis involved evaluating the methodologies employed in these studies and comparing their achieved accuracy rates.

The existing methodologies in these studies included techniques such as XGBoost (XGB), multiple linear regression (MLR), random forests (RFs), support vector machines (SVMs), k-nearest neighbors (KNN), 1D convolutional neural networks (1D CNNs), deep neural networks (DNNs), and bidirectional gated networks (BDGNs). Previous research has demonstrated lower accuracy rates and methodologies that may not fully address the evolving needs of cognitive workload classification. However, the DGNN model stands out prominently in this analysis, showcasing the highest accuracy among the compared studies. Specifically, our model achieves an impressive accuracy rate of 99.45% in accurately classifying cognitive workload levels using EEG data. Figure 11 represents the plotted comparison of state-of-the-art results from Table 1, illustrating the superior performance of our DGNN model relative to these existing methodologies.

4.2. Cognitive Workload Monitoring System

The real-time monitoring of the CWMS is evaluated using the Things Board platform, allowing us to track various parameters, including end-to-end latency, model classification processing time, and cognitive workload levels. The Raspberry Pi Zero W gateways efficiently manage data transmission between the EEG device and the AI server without significant delays, ensuring smooth operation. These modules are essential in this setup due to their cost-effectiveness, making them ideal for large-scale experiments involving multiple users and allowing for scalable deployments. Their small size ensures easy integration into experimental setups, particularly with wearable EEG devices, keeping the setup portable and unobtrusive. Additionally, the built-in Wi-Fi functionality enables seamless wireless communication between the virtual users and the Ubuntu server. The Things Board platform provides a user-friendly interface for visualizing and analyzing the results, with real-time tracking capabilities crucial for monitoring the system’s performance.

Our experiment involved five laptops, with four representing virtual users and one for the live monitoring and classification results. Additionally, four Raspberry Pi Zero W gateways are used to facilitate data transmission between the EEG device and the AI server. To accommodate the connectivity and power needs of Raspberry Pi modules, a single Wi-Fi router and power cables are used. Figure 12 illustrates the experimental setup, depicting the interconnected components.

A set of monitoring data is presented, where each figure comprises three distinct graphs and a table representing the data from four gateways. In our scenario, each gateway serves only one virtual user; thus, the four gateways correspond to four virtual users. The first graph in each figure displays end-to-end latency, providing insights into the transmission delays. The second graph showcases AI model classification processing time, indicating the duration taken for cognitive workload assessment. Lastly, the third graph delineates workload levels, where 0 signifies a low workload level, 1 denotes an average workload level, and 2 indicates a high workload level. The corresponding table in each graph offers a direct representation of workload classification.

Figure 13 displays the information from gateway-001 on Things Board. We observe the latest latency recorded as 237 ms, indicating the time taken for data transmission. The classification processing time is noted as 4621.21 ms, reflecting the duration for cognitive workload assessment. Additionally, the workload levels are showcased separately, providing a comprehensive overview of the system’s performance. This detailed visualization aids in identifying potential issues and optimizing the network’s efficiency.

Figure 14 presents the monitoring data from gateway-002 on Things Board, encompassing three critical graphs and a comprehensive table. The first graph indicates an end-to-end latency of 459 ms, a metric that measures the time taken for the data to travel from the virtual user to the server and back, reflecting the network’s efficiency. Higher latency can adversely affect user experience, particularly in real-time applications. The second graph showcases the classification processing time, recorded at 6717.97 ms, which is significantly higher than that of gateway-001, suggesting potential variability in computational load or hardware performance across different gateways. The third graph delineates the workload levels for gateway-002, distinguishing between low (0), average (1), and high (2) workload levels.

Figure 15 displays the data from gateway-003 on Things Board, following the same structure as the previous figures but with distinct performance characteristics. The latest recorded latency for gateway-003 is 196 ms, which is notably lower than that of gateway-002, indicating a more efficient data transmission process and enhancing the responsiveness of applications relying on this gateway. The classification processing time for gateway-003 is measured at 3849.74 ms, which is substantially shorter compared to gateway-002. This suggests that gateway-003 may have more efficient processing capabilities or less computational load at the time of measurement. The workload levels graph for gateway-003 shows the distribution of cognitive workload levels, providing a visual representation of the virtual user’s cognitive load.

Figure 16 represents the monitoring data from gateway-004 on Thing Board, showcasing its unique performance metrics. The latest latency recorded for gateway-004 is 132 ms: the lowest among all four gateways. This minimal latency suggests that gateway-004 provides the most efficient data transmission, which is critical for applications requiring real-time data processing.

The classification processing time for gateway-004 stands at 2866.82 ms, which is also the shortest among the four gateways. This efficiency in processing indicates a robust performance in cognitive workload assessment, potentially due to better hardware or lower concurrent computational demands. The third graph in Figure 16 illustrates the workload levels for gateway-004, continuing the pattern of distinguishing between low, average, and high workload levels. The table in this figure offers a clear and concise representation of the workload classifications, aiding in performance evaluation and decision-making.

We have proposed a system that enables continuous data transmission, real-time classification, and effective monitoring of results by integrating the DGNN model with Raspberry Pi, CoAP, and Things Board. While we have utilized the STEW dataset in this study, validating our model with real human participants is crucial. We plan to recruit healthy participants from the university, adhering to ethical guidelines and obtaining informed consent. EEG data will be collected during cognitive tasks, and the system’s performance will be evaluated based on classification accuracy, sensitivity, specificity, and response time.

5. Conclusions

This study introduces a novel approach for cognitive workload classification using EEG signals and advanced deep learning techniques. Leveraging the STEW dataset, comprehensive preprocessing and correlation analyses were conducted, followed by detailed short-time Fourier transformation analysis to extract relevant features. Our proposed deep gated neural network (DGNN) model, integrating Bi-LSTM and GRU layers, achieved an impressive accuracy of 99.45%. Precision, recall, and F1 score metrics further validated the efficacy of our model in effectively classifying cognitive workload across distinct categories. A comparative analysis with existing methodologies underscores the superior performance and robustness of the DGNN model in this domain.

To validate the proposed approach, this study conducted the cognitive workload monitoring experiment (CWME), deploying the DGNN model on a classification server. The deployment leveraged Raspberry Pi Zero W modules as gateways, employing the constrained application protocol (CoAP) for efficient data transfer to the Things Board monitoring platform. Things Board facilitated real-time visualization and analysis of critical parameters such as end-to-end latency, AI model classification processing time, and workload levels, providing actionable understandings into system performance. This setup enhances operational efficiency by enabling prompt cognitive workload assessments, supporting dynamic decision-making processes. The proposed research provides practical solutions for managing cognitive workload in industrial settings amidst ongoing digital transformation and automation [69].

The successful implementation and validation of the DGNN model in real-time cognitive workload monitoring hold significant implications for various fields, particularly in industrial settings. By accurately assessing cognitive workload levels using EEG signals, organizations can proactively manage workforce health and safety. This approach enables timely interventions to mitigate cognitive fatigue, thereby reducing the risk of errors and accidents in high-risk environments. Moreover, the ability to monitor workload levels in real-time fosters adaptive work environments, potentially enhancing overall productivity and employee well-being.

Despite the promising results achieved in this study, transitioning from virtual users to real human participants poses several significant challenges. Variability in EEG signal quality and ensuring participant comfort with wireless headsets are critical considerations that can impact data accuracy and participant engagement. Adhering to study protocols and ethical guidelines for EEG data collection is essential to maintain data integrity and participant trust. Additionally, the wide cost range of wireless EEG devices, spanning from 150 to 30,000 USD, and the variability in battery life (3 h to 10 h) present financial constraints and usability issues that must be carefully managed [70,71].

6. Future Works

While this study has laid a solid foundation for real-time cognitive workload monitoring using EEG signals and advanced deep learning techniques, several avenues for further research and development are essential to enhance the applicability and effectiveness of our approach in practical settings. A crucial next step involves validating our proposed deep gated neural network (DGNN) model with real human participants. While virtual users provide valuable insights, incorporating real-world factors such as individual EEG variations, stress levels, and environmental influences will enhance the model’s robustness and applicability across diverse work environments. Longitudinal studies involving diverse participant groups will be essential to evaluate the model’s performance under varying cognitive workload conditions over extended periods.

Improving the usability of wireless EEG headsets represents another significant avenue for future research. Current limitations, such as discomfort during prolonged wear and restricted mobility, need to be addressed through innovative ergonomic designs and lightweight materials. Enhancing headset ergonomics to minimize discomfort and fatigue, alongside optimizing sensor placement for better signal quality, will be crucial for facilitating long-term, real-world deployment. Future research should focus on scaling the DGNN model to accommodate multiple users concurrently and evaluating its performance under diverse operational conditions and noise levels. Assessing the system’s robustness and reliability across different industrial contexts will be essential for broader adoption and practical implementation.

Additionally, integrating real-time alarm systems into our monitoring platform using tools like Things Board offers an opportunity to enhance responsiveness to cognitive workload fluctuations. Implementing proactive alert mechanisms based on workload classifications will empower timely interventions and adaptive decision-making, thereby improving operational efficiency and safety in dynamic settings. Addressing the challenges ahead will be crucial for fully realizing the potential of our approach in practical, real-world applications.