A Machine Learning Approach Towards the Quality Assessment of ECG Signals Collected Using Wearable Devices for Firefighters

Abstract

This work focuses on assessing the ECG signal quality of data collected with wearable devices specifically tailored for firefighters using machine learning techniques. Firefighters are at a heightened cardiac risk due to their challenging working conditions, making wearable sensors crucial for ongoing health monitoring. However, environmental factors such as the temperature, radiation, and moisture, significantly impact the performance of these sensors and the quality of the collected data. To address these challenges, this work explored supervised learning to classify ECG signals into acceptable and unacceptable segments using only eight cardiac features. Leveraging on the ScientISST MOVE dataset, which contains biosignals during various daily activities, the model achieved promising results, namely 88% accuracy and an 87% F1 score with just eight ECG features. Besides this, a case study was performed on ECG data gathered from firefighters under real-world conditions to further corroborate the proposed method. Such a validation exercise demonstrated how well the model performs for the assessment of signal quality in such dynamic, high-stress scenarios.

1. Introduction

1.1. Risks Faced by First Responders

Every day, First Responders (FRs; e.g., firefighters, civil protection units, police, etc.) face immense physical and environmental challenges during their duties. Their high-risk roles expose them to toxic fumes, extreme exertion, heat stress, prolonged loud noise, and long work hours. These conditions can lead to hypoxia, cellular ischemia, dehydration, hypertension, and coronary artery disease, significantly increasing the risk of acute cardiac events. In fact, the National Fire Protection Association reported that 44% of firefighter deaths are due to sudden cardiac events [1].

Given these risks, continuous health monitoring is of the utmost importance. Wearable ECG sensors can provide the real-time tracking of heart health, allowing for the early detection of potential issues and improving the overall safety. This proactive approach enhances both the safety and effectiveness of FRs in critical situations [2].

1.2. Challenges in Wearable Technology for Firefighters

To date, despite the well-known cardiac risk faced by on-duty firefighters, especially during emergency activities, and its societal importance, there are no available ECG field data recordings during firefighting activities. This is due to external constraints imposed by environmental conditions and/or the effect of personal protective equipment, such as the temperature, radiation, and moisture. Radiation can cause sensor errors, while moisture can decrease the sensor lifespan and restrict long-term monitoring.

ECG sensors face significant challenges due to the influence of noise from both the surrounding environment and the human body (e.g., interference from physical movement and breathing), complicating accurate analysis, especially in dynamic conditions. Previous studies have been conducted to explore the integration of ECG-enabled smart textiles into firefighters’ gear [3].

A representative effort was the Vital Responder project, which aimed to optimize firefighting efforts and minimize fire damage by developing the VitalJacket, a discreet device capable of monitoring firefighters’ cardiovascular status through a one-lead clinical ECG without restricting movement [4]. However, adapting wearable technologies to challenging environments like forest fires presents challenges, resulting in poor-quality ECG signal acquisition due to damaged electrodes, hardware issues, and incorrect time annotations of the data. Another challenge is complying with international safety regulations for textiles in high-heat conditions, as seen in this study. For example, an initial version of the VitalJacket contained over 20% elastane, which ensures good electrode compression against the skin but is heat-sensitive and could potentially cause burns. This is why garments composed of 2% elastane or more are not allowed by European regulations for use in fire contexts due to the associated risk of harming the wearer’s skin [5].

Modern firefighting clothes are made from high-tech fire-resistant or thermal-resistant fibers, offering durability and excellent mechanical and abrasive characteristics. Aramid, a popular material in fire-protective clothes and military uniforms, is known for its heat resistance, mobility retention, and lightweight, breathable, and thermally insulating properties (https://www.tchaintech.com/CDETAIL/case-firefighter-aramid-fabric (accessed on 20 November 2024)).

A smart undergarment for firefighters is highly valuable for its ability to integrate sensors directly into the fabric to monitor vital signs, stress levels, and physical activity. The integration of sensors into the undergarment also allows for the measurement of physiological signals directly on the firefighter’s body, providing precise real-time information to monitor health and safety during firefighting operations [6].

However, a t-shirt undergarment is not suitable for efficient ECG collection during firefighting tasks due to its flexibility and lightness. Additionally, the use of dry electrodes, rather than hydrogel ones, to prevent skin irritation introduces more noise into the ECG signal. Consequently, the increased difficulty in achieving electrode adherence to the skin exacerbates the issue of electrode contact loss [3,5]. A sports vest could provide a tight fit without conditioning firefighters. The EquiVital project (https://equivital.com (accessed on 20 November 2024)) offers a similar idea but has drawbacks like lycra electrodes oxidating over time.

Another challenge in implementing a smart undergarment is the movement of the skin across the electrodes, which introduces noise into the ECG signal [2], and electrode folding and stretching due to firefighters’ movements.

Research on ECG signal quality assessment (SQA) is primarily focused on clinically acquired signals, not those from wearables. This may not directly address noise under dynamic conditions. With the increasing portability of ECG measurement devices and the need to process large volumes of data contaminated by noise and motion disturbances, developing techniques to detect segments of interest is of utmost importance.

1.3. Related Work

1.3.1. ECG Signal Quality Assessment Overview

ECG SQA is crucial for improving accuracy in automated analysis systems. Traditional systems often give incorrect measurements due to noisy signals, leading to high false alarm rates. A signal quality index (SQI) approach can be used alongside noise reduction strategies to assess the reliability of ECG signals in noisy conditions. SQIs can be based on signal processing techniques, machine learning (ML), and/or decision rules. In previous work, the ECG signal quality has been categorized into five groups (excellent, good, adequate, poor, and unacceptable), three groups (acceptable, indeterminate, and unacceptable), or two groups (acceptable and unacceptable), with the two-group classification being the most used [7].

J. Behar et al. [8] used three datasets (PhysioNet/CinC Challenge 2011, MIT-BIH Arrhythmia Database, and MIMIC II Database (

Table 1. Summary of characteristics of ECG datasets.

Dataset	Year	Number of Recordings/Segments	Number of Leads	Quality Annotations	Arrhythmia Annotations	Additional Information
PhysioNet/CinC Challenge 2011 1	2011	1539 of 10 s	12	“Acceptable”, “Unacceptable”	None	Sampled at 500 Hz with 16-bit resolution.
PhysioNet/CinC Challenge 2017 2	2017	8528 from 30 to 60 s	1	Indication of noise presence or absence	AF, Normal Sinus Rhythm (NSR), Other Rhythm (OR)	Collected with an AliveCor hand-held ECG device; sampled at 300 Hz and bandpass-filtered by the AliveCor device.
MIT-BIH Arrhythmia 3	1980	48 of 30 min	2	Indication of noise presence or absence	15 arrhythmias	Ambulatory ECG recordings were digitized at 360 samples per second.
MIMIC II 4	2007	25,328 of different lengths	Not specified	Indication of noise presence or absence	AF, NSR, OR	Sampled at 125 samples per second.
China Physiological Signal Challenge 2020 5	2020	10,330	12	Indication of noise presence or absence	AF, NSR, OR	Collected by a unified wearable ECG device with a sampling frequency of 400 Hz.
Lenovo [10]	2020	9000 of 10 s	3	Indication of noise presence or absence	AF, NSR, OR	Collected with a Lenovo H3 wearable ECG device at a sampling rate of 400 Hz.
Sensecho [11]	2021	8146 of 10 s	Not specified	“Good”, “Acceptable”, “Unacceptable”	None	Collected by a SensEcho wearable device.
ScientISST MOVE [12]	2024	17 participants, avg. of 37 min each	1	Not specified	Not specified	Data collected using the ScientISST device during everyday activities, with a sampling frequency of 500 Hz. Includes annotations of everyday activities.

¹ https://physionet.org/content/challenge-2011 (accessed on 20 November 2024). ² https://physionet.org/content/challenge-2017 (accessed on 20 November 2024). ³ https://physionet.org/content/mitdb (accessed on 20 November 2024). ⁴ https://archive.physionet.org/mimic2/ (accessed on 20 November 2024). ⁵ https://physionet.org/content/challenge-2020/1.0.2/sources/ (accessed on 20 November 2024).

)) to assess the quality of an ECG for normal and abnormal rhythms. Their study used various SQIs, including the relative power in the QRS complex (pSQI); the relative power at the baseline (basSQI); the skewness of the distribution (sSQI); the kurtosis of the distribution (kSQI); the percentage of matching beats detected by the eplimited and wqrs algorithms (bSQI) [9]; the ratio of the number of beats detected by the eplimited and wqrs algorithms (rSQI); and the ratio of the sum of the eigenvalues associated with five principal components to the sum of all the eigenvalues (pcaSQI). A Support Vector Machine classifier was used for classification, achieving 99.0% accuracy (Ac), 98.5% sensitivity (Se), and 99.4% specificity (Sp).

1.3.2. ML-Based Methods for SQA

J. Kuzilek et al. [13] used a three-stage algorithm to filter low-quality ECG signals using the PhysioNet/CinC Challenge 2011 dataset (Table 1). Initially, basic features like the variance, covariance, maxima, dynamic range, and averages were computed and compared to a threshold to assign a score. Additional features like the time-lagged covariance metrics, mean, kurtosis, and QRS complex count were processed in the second stage and used as the input to an SVM. The final assessments of the signal quality were combined, resulting in 99.9% for the training set and 83.6% for the test set.

In [14], the authors developed a single-channel ECG quality metric using six SQIs (pSQI, kSQI, basSQI, bSQI, rSQI, and pcaSQI) and a SVM classifier. To address a data imbalance, noisy data were generated by adding noise from the Noise Stress Test Database to clean records from the PhysioNet/CinC Challenge 2011 and MIT-BIH Arrhythmia Database (Table 1). The system was trained to classify ECG segments as clean or noisy, ensuring that only high-quality data were considered suitable for clinical use. The best results on the extended database were an Ac of 97.9%, Se of 97.6%, and Sp of 98.1% on the training set and an Ac of 97.1%, Se of 97.7%, and Sp of 96.5% on the test set.

Qiao Li et al. [15] trained an SVM to classify the ECG signal quality into five classes using 13 SQIs. This approach aimed to provide a more detailed noise level classification for various clinical applications, surpassing the traditional binary classification of clean or noisy ECG signals. The results showed a classification Ac of 80.26% and an overlap accuracy (OAc) of 98.60% on the test set using 10 selected metrics. For unseen MIT-BIH Arrhythmia Database (Table 1) validation data, the Ac was 57.26% and the OAc was 94.23%. Fivefold cross-validation yielded an Ac of 88.07% ± 0.32% and an OAc of 99.34% ± 0.07%.

1.3.3. Deep Learning for SQA

C. Ma et al. [16] developed three deep ML-based ECG signal quality assessment models using the xResNet architecture. They proposed a model that accurately classifies the ECG signal quality, even in arrhythmias and noisy signals, using the China Physiological Signal Challenge 2020 (CPSC 2020) dataset (Table 1), which contains 24 h dynamic ECG recordings from 10 patients with arrhythmias. The proposed models were designed to distinguish between the following:

• ECG data with visible PQRST waves or visible QRS complexes only and data where these features cannot be utilized for analysis.
• ECG data with visible PQRST waves.
• ECG data classified into categories of visible PQRST waves, visible QRS waves only, or unsuitable for analysis.

The xResNet34-based SQA algorithm outperformed previous methods using decision trees and manually extracted features, achieving average accuracies of 96.62%, 93.66%, and 98.97% for each category, respectively.

Huerta et al. [17] classified ECG intervals as high- or low-quality using a pre-trained AlexNet Convolutional Neural Network (CNN). The authors used the PhysioNet/CinC Challenge 2017 dataset (Table 1) for the classification, selecting 2.000 five-second segments of ECGs (1.000 high-quality and 1.000 low-quality) and converting them into matrices using a Continuous Wavelet Transform. Despite the advantages of deep learning (DL), including automatic feature extraction and less need for specialized annotations, there were no statistically significant differences in the classification results between the original and augmented datasets. The performance metrics averaged around 90% with a less than 6% standard deviation across five iterations.

1.3.4. Wearable Device Integration

In [18], Wang aimed to develop an automated algorithm for assessing ECG signal quality using supervised ML. In this study, two datasets were collected using wearable devices, one involving six subjects using an ECG belt under controlled conditions and the other using four subjects using an ECG patch under natural conditions. The goal was to estimate the ECG signal quality for better heart rate monitoring and cardiac diagnosis. The study compared various feature extraction methods (wavelet transforms, an AutoCorrelation Function (ACF), and Principal Component Analysis (PCA)) using a 5-fold cross-validation classification model. After feature selection, an SVM was trained on 10 s segments labeled by experts. The discrete wavelet transform outperformed the others, enabling an Ac of 92.5% and an AUC of 94.8%, followed by PCA and the ACF.

F. Fu et al. [10] conducted a study on ECG SQA using data from the Lenovo H3 wearable ECG device, developed by their laboratory and Lenovo (Table 1). They classified ECG signals into two categories, namely “acceptable” and “unacceptable”, where unacceptable signals could be affected by environmental noise, electrode displacement, and other artifacts. They analyzed 9.000 ten-second one-lead ECG segments, labeled by cardiologists, using non-morphological quality indices such as the approximate entropy, sample entropy, and fuzzy measure entropy. The Long Short-Term Memory classifier achieved the highest Ac of 95.8% on these segments.

H. Xu et al. [11] validated an SQA algorithm on ECG and respiratory signals from a SensEcho wearable (Table 1) using semi-supervised ML with an Isolation Forest. The algorithm categorized 10 s ECG segments into good, acceptable, and unacceptable levels, achieving an accuracy of 94.97% and 95.58% for ECGs in the validation and test sets, respectively, surpassing the Self-Organizing Map model and performing moderately compared to traditional supervised models.

1.4. Summary

Advancements in ECG SQA have demonstrated promising results, particularly in controlled environments. Traditional ML methods, such as SVM classifiers with handcrafted SQIs (e.g., pSQI, bSQI), achieve high accuracy (up to 99%) by leveraging noise augmentation and multi-stage pipelines. DL approaches, including CNNs and xResNet architectures, further automate feature extraction, attaining state-of-the-art performance (e.g., 98.97% accuracy for multi-class SQA). However, critical limitations persist. First, models often overfit to curated datasets (e.g., PhysioNet challenges), resulting in poor generalization to unseen data (e.g., 57% accuracy on MIT-BIH). Second, reliance on short, static ECG segments and synthetic noise simulations fails to address the dynamic noise profiles inherent in wearable devices, such as motion artifacts during physical activity.

Methodological trade-offs exist between interpretability (traditional ML) and performance (DL), with the latter posing challenges for real-time deployment on resource-constrained wearables. Dataset heterogeneity further complicates reproducibility, as studies often use device-specific or narrowly annotated data (e.g., Lenovo H3 wearable ECG device, SensEcho (Table 1)), limiting cross-domain applicability. While emerging datasets like ScientISST MOVE are beginning to address dynamic conditions, their integration into SQA frameworks remains underexplored.

1.5. Objectives and Structure

This study bridged these gaps by focusing on wearable ECG signals under real-world, dynamic scenarios and leveraging activity-annotated datasets to improve robustness. The main aim was to contribute to the identification of features that can effectively assess usable segments within an ECG signal, particularly under the dynamic conditions typical of wearable devices. This area remains underdeveloped, yet it holds significant importance in today’s world. A supervised model was designed to classify whether an ECG segment is acceptable (free from noise) or unacceptable (contains noise).

The paper is structured as follows: Section 1.2 addresses the challenges of wearable technology for firefighters and the limitations of smart undergarments. Section 1.3 offers a review of the existing literature concerning ECG quality assessment algorithms, with a focus on wearable ECG signal assessment. Section 2 details the Materials and Methodology, including the characterization of the ScientISST MOVE dataset [12] (Section 2.1) and the use of the Python BioSPPy library [19] for processing ECG signals and generating ground truth labels (Section 2.2); feature extraction for model training (Section 2.4) and parameter optimization for classification models are also described (Section 2.5). Additionally, Section 2.6 presents a case study with firefighters under real conditions, demonstrating the effectiveness of the algorithm in practical scenarios. The results are presented in Section 3, followed by the Discussion in Section 4. Section 4.1 addresses the performance of the ECG assessment model, Section 4.2 examines the features used, and Section 4.3 discusses the analysis of ECG signals collected from firefighters in real-life conditions, emphasizing the importance of an ECG assessment algorithm that prioritizes quality. Lastly, Section 5 outlines the main conclusions.

2. Materials and Methods

2.1. ECG Dataset

The development of robust ECG segmentation algorithms faces challenges, as highlighted in [17], due to a lack of publicly available databases with annotated wearable ECG samples. Furthermore, few datasets incorporate both acceleration data and a comprehensive set of five physiological signals. The inclusion of acceleration data is particularly valuable for identifying periods of high-intensity activity, during which more noise is introduced into the ECG signal. Another challenge arises from the variability in electrode types (dry or gel) and their placement on different parts of the body, which can affect the signal quality and consistency.

To address this, in this work, we used the ScientISST MOVE dataset (Table 1) [12], a collection of wearable biosignals used to study everyday activities, such as lifting a chair, greeting gestures, jumping, walking, and running. It includes data from three wearable devices: a ScientISST chest band, a ScientISST armband, and an Empatica E4 wristband. These devices monitor physiological signals, including Electrodermal Activity [20], Photoplethysmography [21], Electromyography [22], and ECGs, and also 3-axis accelerometer data.

The study collected data from 17 healthy individuals, each contributing an average of 37 min of synchronized sensor recordings, with the recordings from 11 participants containing pre- and post-running walking periods. This subset yielded 563.8 min of ECG data, averaging 51.25 min per participant, allowing for an examination of ECG signal variations across different physical activities. In Figure 1, we can observe differences in the ECG signals for these different activities, demonstrating that some activities introduce more noise than others, such as jumping and lifting weights. All data were collected from healthy volunteers, none of whom had a history of cardiovascular disease, such that the noise to be analyzed was primarily caused by environmental and motion-related sources.

The study used an ECG sensor connected to a Polar chest band with dry conductive textile contacts, mimicking smart undergarments’ electrode conditions, enabling increased noise sensitivity in ECG signal capturing. The chest band comprised two electrodes.

2.2. Pre-Processing of ECGs

The ECG sensor used in the ScientISST MOVE dataset amplifies cardiac signals with a gain of 1100 and operates at 3.3 V, enabling a ±1.5 mV measurement range, making it sensitive to subtle cardiac changes. Its input impedance of 100 GΩ minimizes signal loading, and a 110 dB Common-Mode Rejection Ratio contributes to low noise and interference.

The pre-processing of the ECG signal involved several key steps (Figure 2). First, the ECG data, measured in millivolts (mV), were extracted from an .edf file using the pyedflib.EdfReader library in Python. If the signal polarity was reversed, it was corrected through multiplication by −1.

Following extraction, the signals, originally sampled at 500 Hz, were downsampled to 100 Hz. This step reduced both the data size and computational complexity, enhancing the efficiency of the algorithm and making it more practical for real-world applications.

After downsampling, the biosppy.signals.ecg.ecg function (https://github.com/scientisst/BioSPPy/blob/main/biosppy/signals/ecg.py (accessed on 20 November 2024)) from the BioSPPy library [19] was applied to the signal. This function processed the ECG by applying a bandpass Finite Impulse Response (FIR) filter between 0.67 Hz and 45 Hz, removing baseline wander, high-frequency noise, and motion artifacts. It also eliminated the DC offset by subtracting the mean of the filtered signal, ensuring signal stability. The function output the filtered ECG signal, which was then used for further processing and model training.

Finally, normalization was performed by scaling the signal based on the range between its maximum and minimum values and subtracting the minimum value within a 10 s segment.

These pre-processing steps guaranteed that the ECG signal was clean and prepared for further analysis, such as feature extraction.

To train the supervised ML model, the ground truth for ECG signals, denoted as 0 (unacceptable) or 1 (acceptable), depicted in Figure 3, was determined through a two-step process:

• Data Analysis (Assessment Criteria) [11]:
  • Acceptable (1):

    -

    The ECG rhythm is clear; each QRS waveform can be distinguished with the naked eyes.

    -

    Low-intensity high-frequency noise; the R waves in the signal can be recognized accurately.

    -

    No more than 2 high-frequency impulse noises occur in the observation window.

  • Unacceptable (0):

    -

    Full of noise.

    -

    More than 2 R-peaks in the observation window cannot be distinguished.

• Application of the BioSPPy Library: In order to obtain the ground truth, the existing function—called ecg.ecg—from the BioSPPy library was used to change the signal before proceeding with the description of the various cardiac features. The main purpose of using this function was to guarantee that the signal would be deemed of sufficiently high quality for the robust calculation of the cardiac features. In cases of poor signal quality, the corresponding features would be replaced by standard established values.
  • If the function ecg from the BioSPPy library returned the error “Not enough beats to compute heart rate” for an ECG segment, this indicated that the signal did not have sufficient recognizable QRS complexes to compute the heart rate. In such cases, the ECG segment was automatically marked as unacceptable (0).
  • If this error did not occur, it indicated that the signal was sufficiently clean for cardiac feature extraction. In this scenario, the ECG signal was categorized into the appropriate quality level based on the predefined assessment criteria.

After establishing the intervals that were 0 and 1 in each ECG signal, to indicate whether a segment of a certain length was considered acceptable or unacceptable, the mode of the segment’s quality values was calculated.

2.3. Computational Environment

To ensure efficient data processing and model training, all ML experiments were conducted on a system equipped with an AMD Ryzen 7 5700U processor, 16GB RAM, and an NVIDIA RTX 3060 GPU.

The software environment was implemented using Python 3.9, with key libraries including the following:

• PyEDFLib: Used for reading .edf files containing the raw ECG recordings.
• Scikit-learn: Used for feature scaling, model training (RF classifier), cross-validation (Stratified K-Fold), and performance evaluation (accuracy, precision, recall, and F1 score).
• NumPy and Pandas: Used for the efficient data handling, segmentation, and statistical analysis of ECG signals.
• Matplotlib and Seaborn: Applied for data visualization, including the creation of ECG signal plots and feature distributions.
• BioSPPy: Utilized for ECG pre-processing, R-peak detection, and feature extraction.

2.4. ECG SQIs and Other Features

For the ML classification model of the ECG signal, features were extracted and divided into two domains: time and SQIs (

Table 2. Extracted ECG signal features for model training, divided into two domains: time and SQIs.

Domain	Feature	Explanation
Time	Standard deviation of RR intervals (SDRR)	Standard deviation of the intervals between consecutive R-peaks.
	Variability of R-peaks compared to maximum amplitude (MaxRR)	Ratio of the maximum amplitude of R-peaks to the maximum amplitude of the signal.
	Inter-beat variability (InterV)	Average standard deviation of each time point across all heartbeat templates, quantifying the variability between different heartbeat patterns.
	Intra-beat variability (IntraV)	Average standard deviation computed within each heartbeat template, quantifying the variability in the shape or pattern of each beat.
	Flatline percentage (FlatP)	Percentage of the signal where the amplitude is below a certain threshold.
SQI	Quality	Numeric representation of overall ECG quality based on multiple SQIs (qSQI, pSQI, kSQI, and basSQI).
	hosSQI	Combination of the skewness quality index absolute value (sSQI) and the kurtosis quality index (kSQI).
	cSQI	Ratio of the standard deviation to the mean of RR intervals.

).

Regarding the time domain features, the standard deviation of the RR intervals (SDRR) was calculated. Additionally, the variability of the R-peaks relative to the maximum amplitude of the signal (MaxRR) was determined by calculating the ratio of the maximum amplitude of the R-peaks to the maximum amplitude of the signal. Different types of variability in heartbeat templates were evaluated, including intra-beat variability (IntraV) and inter-beat variability (InterV). IntraV measures the variability within each individual beat by first calculating the standard deviation for each beat and then averaging these standard deviations (Figure 4). This approach captures the variability in the shape or pattern of each beat. On the other hand, InterV assesses the variability between beats by stacking all the beats together, calculating the standard deviation at each time point across beats, and then averaging these standard deviations. This reflects the variability among different beats as a whole. Finally, the flatline percentage (FlatP) was computed, indicating the percentage of the signal that fell below a certain threshold.

The “Quality” feature in the SQIs was derived from the BioSPPy library using the ZZ2018 function, categorizing the signal quality as “Excellent”, “Barely acceptable”, or “Unacceptable”. These quality levels were numerically mapped for training the model.

The SQIs involved in the “Quality” feature included qSQI, which assessed the agreement between two peak detectors, and pSQI, which calculated the ratio of the energy of the QRS complex to the total energy of the ECG signal [23]. If there was noise interference, the high-frequency component increased and pSQI decreased. It also included kSQI, which calculated the kurtosis of the ECG signal, and basSQI, which computed the complementarity of the power ratio in the frequency bands from 0 to 1 Hz to the power in the frequency bands from 0 to 40 Hz. The basSQI measured the effect of baseline drift noise reduction in ECG signals [24]. The analysis used a fuzzy mode to calculate the quality measure by combining the weighted sums of the four SQIs, chosen due to its superior Ac compared to other methods [24].

The cSQI [24] assessed the variability in the RR intervals by calculating the ratio between the mean of the RR intervals and their standard deviation. The equation is given by:

$$\mathrm{cSQI}={\displaystyle \frac{{\hat{\sigma }}_{RR}}{{\hat{\mu }}_{RR}}},$$

(1)

where ${\hat{\sigma }}_{RR}$ and ${\hat{\mu }}_{RR}$ are the empirical estimates of the standard deviation and mean of the distribution of the RR intervals, respectively. In general, a cSQI value closer to 0 indicates a more regular heart rate, while higher values indicate more variability, ranging from 0 to infinity.

Finally, a higher-order statistics SQI (hosSQI) [23] was derived from two SQIs: sSQI, which computed the skewness of the signal; and kSQI. It is computed as follows:

$$\mathrm{hosSQI}=\left|sSQI\right|\times {\displaystyle \frac{kSQI}{5}}$$

(2)

Since skewness can be both positive and negative, the absolute value was taken to ensure non-negativity. The skewness can range from negative infinity to positive infinity. However, typical values for a healthy ECG signal would be within a certain range, often between −1 and 1. Regarding kurtosis, a good-quality ECG signal generally has a kSQI of >5 [25]. Therefore, the hosSQI can theoretically range from 0 to infinity.

These two indices were the third (sSQI) and fourth (kSQI) statistical moments and were used to characterize a signal distribution, also evaluating the level of non-Gaussianity in the case of correlated signals.

The model’s discriminative power and variance sources were assessed by computing normalized averages for each metric, comparing ECG segments categorized as 0 (unacceptable) or 1 (acceptable).

2.5. Parameter Optimization and Performance Evaluation

The Random Forest (RF) was chosen for classification due to its robustness and suitability for noisy data. To optimize performance, two parameters were considered: the ECG segment lengths; and overlap. Four metrics were calculated, namely the Ac, precision, Se, and F1 score [26], with TPs, TNs, FPs, and FNs representing true positives, true negatives, false positives, and false negatives, respectively (Equations (3)–(6)).

$$\mathrm{Accuracy}={\displaystyle \frac{TP+TN}{TP+FN+FP+TN}}$$

(3)

$$\mathrm{Precision}={\displaystyle \frac{TP}{TP+FP}}$$

(4)

$$\mathrm{Recall}=\mathrm{Sensitivity}={\displaystyle \frac{TP}{TP+FN}}$$

(5)

$$\mathrm{F}1\mathrm{score}={\displaystyle \frac{2\times \mathrm{Precision}\times \mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recall}}}$$

(6)

The study employed an RF classifier for stratified 5-fold cross-validation, as proposed in [27].

2.6. Case Study: Firefighters Under Real Conditions

ECG signals were gathered from tests conducted in real-life situations involving firefighters from various corporations in Portugal, demonstrating the effectiveness of this algorithm. The signals were recorded using a sports vest equipped with integrated e-textile dry electrodes linked to a MoveSense device to measure the ECG signal. Additionally, survey tools were administered to assess the challenges and conditions of daily activities, as well as the physical and mental well-being of the firefighters. These data were then compared under dynamic conditions, such as days with varying levels of activity and instances of mental or physical fatigue.

The ECG signals were recorded continuously during the firefighters’ 8 h shifts, with each participant wearing the vest equipped with the MoveSense device (https://www.movesense.com/movesense-md-wearable-ecg-monitor/ (accessed on 20 November 2024)). At the end of each shift, participants reported their perceived activity levels and physical condition by filling out a questionnaire.

The ML quality assessment algorithm was applied to the ECG data, classifying each 10 s segment as either 0 (unacceptable) or 1 (acceptable). The percentage of 10 s segments considered “acceptable” was then calculated for each 8 h shift.

3. Results

The model’s performance metrics were analyzed using different ECG segment lengths and overlap percentages (

Table 3. Performance metrics (accuracy, precision, recall, F1 score) for different combinations of interval lengths (seconds) and overlap percentages in the ECG signal segmentation. The highlighted cells indicate the best combination of the interval length and overlap.

		Interval Length (Seconds)
Metric	Overlap	5	6	7	8	9	10
Accuracy	50%	0.860	0.853	0.869	0.858	0.865	0.879
	60%	0.857	0.867	0.858	0.862	0.871	0.871
	70%	0.855	0.865	0.859	0.865	0.872	0.867
	80%	0.863	0.859	0.861	0.864	0.858	0.866
	90%	0.862	0.865	0.854	0.862	0.862	0.863
Precision	50%	0.896	0.902	0.909	0.899	0.907	0.918
	60%	0.902	0.910	0.910	0.904	0.906	0.910
	70%	0.897	0.909	0.907	0.906	0.907	0.907
	80%	0.904	0.905	0.906	0.906	0.902	0.906
	90%	0.902	0.905	0.907	0.904	0.903	0.903
Recall	50%	0.830	0.812	0.838	0.827	0.831	0.844
	60%	0.819	0.828	0.812	0.829	0.846	0.838
	70%	0.819	0.826	0.819	0.831	0.846	0.835
	80%	0.827	0.818	0.823	0.830	0.819	0.832
	90%	0.826	0.827	0.805	0.826	0.827	0.830
F1 Score	50%	0.849	0.836	0.859	0.846	0.852	0.866
	60%	0.843	0.853	0.841	0.849	0.863	0.860
	70%	0.841	0.852	0.845	0.853	0.863	0.855
	80%	0.851	0.845	0.848	0.851	0.843	0.853
	90%	0.850	0.852	0.836	0.849	0.849	0.851

). The segment length refers to the duration of each segment, while the overlap percentage indicates the commonness between consecutive segments. A 50% to 90% overlap ensures continuous analysis by capturing transitional data. The decision to use 10 s intervals was supported by previous studies that have demonstrated the effectiveness of this segment length in balancing the granularity of analysis with computational efficiency. Shorter intervals might not capture enough data for accurate assessment, while longer intervals could introduce unnecessary complexity and delays.

Table 3 details how varying these parameters (segment length and overlap) affected key metrics like the Ac, precision, Se, and F1 score during the assessment of ECG signal quality.

The ScientISST MOVE dataset provided a rich source of data with 6468 ECG segments of 10 s durations with a 50% overlap. Of these, 3.207 segments (49.58%) were classified as unacceptable, while 3.261 segments (50.42%) were deemed acceptable (

Table 4. Distribution of ECG signal quality in the ScientISST MOVE dataset, categorized into segments classified as unacceptable (0) and acceptable (1). The classification was based on 10 s interval lengths with a 50% overlap.

ECG Signal Quality	Number of Segments
Unacceptable (0)	3207
Acceptable (1)	3261
Total	6468

).

Figure 5 shows the differences in the ECG features between acceptable and unacceptable segments.

Figure 6 shows examples of ECG signals labeled as acceptable and unacceptable under three different conditions of dynamic firefighter activities: (1) no activity; (2) the minimum difficulty with favorable conditions and no physiological or psychological fatigue; and (3) moderate difficulty with favorable conditions but with physical fatigue. The proportion of acceptable ECG segments within an 8 h shift for each scenario was 32.6% for (1), 37.8% for (2), and 54.7% for (3).

4. Discussion

4.1. Performance Analysis

The model’s Ac was consistent across interval lengths and overlap settings (Table 3), indicating that the classifier’s ability to correctly label both classes (0 and 1) is not significantly affected by these parameters. The F1 score, the harmonic mean of the precision and Se, followed a similar pattern to that of Se, declining with higher overlaps within longer intervals.

The 10 s interval length with a 50% overlap consistently outperformed other features, achieving the highest scores in terms of Ac (87.85%), precision (91.75%), Se (84.42%), and the F1 score (86.63%), making it the optimal setup for ECG signal segmentation.

Despite expectations, lower overlap intervals showed superior performance, while poorer results occurred with higher overlap percentages and shorter interval lengths, with a 7 s interval and 90% overlap yielding the lowest scores in terms of Ac (85.41%), Se (80.48%), and the F1 score (83.59%).

While these results demonstrate strong classification performance, it is crucial to place them in the context of existing research. However, ECG SQA in wearable conditions remains an underexplored area, making direct comparisons challenging. As highlighted in Section 1.3, none of the reviewed studies incorporated activity-annotated ECG recordings allowing for SQA across different motion contexts.

We compared our findings with the studies described in Section 1.3.4, as these works also focused on ECG quality assessment in wearable systems. Wang [18] achieved an accuracy of 92.5% in a controlled setting using a wearable ECG belt and patch, but their dataset did not include real-world motion noise, limiting its applicability to dynamic environments. F. Fu et al. [10] obtained a higher accuracy of 95.8% using a Lenovo H3 wearable ECG device and LSTM; however, their approach relied on cardiologist-labeled data, whereas our method does not depend on expert-labeled ground truth, making it more adaptable for real-time applications. H. Xu et al. [11] developed an unsupervised ECG quality assessment method for a SensEcho wearable device, achieving 94.97% accuracy with an Isolation Forest algorithm. While effective, their approach primarily focused on anomaly detection rather than ECG quality assessment, which was the key objective of our study.

Despite achieving lower accuracy than some deep learning-based studies, our approach demonstrated strong classification performance with an accuracy of 87.85% while using only eight extracted features. This makes it computationally efficient and well suited for real-time wearable applications. Additionally, unlike previous work conducted under controlled conditions, our study accounted for motion-induced noise and real-world variability, ensuring its relevance for dynamic environments such as firefighter health monitoring. Another key contribution of our work is the analysis of the segment length and overlap percentages, a parameter rarely explored in wearable ECG classification research. By integrating these comparative insights, we reinforce the uniqueness of our study and highlight the need for further research in wearable ECG quality assessment, particularly in high-noise, real-world settings such as firefighter operations.

4.2. Feature Analysis

The ScientISST MOVE dataset revealed “Quality” to be the most distinguishing metric for ECG signals (Figure 5). Acceptable signals have higher “Quality” values, while unacceptable ones have lower values.

The FlatP, IntraV, InterV, SDRR, and cSQI showed a notable negative disparity, implying that superior-quality ECGs exhibit a reduced mean flatline percentage, decreased beat-to-beat variability, and lower RR interval variability in contrast to lower-quality ECGs (Figure 5).

HosSQI and cSQI showed high positive loadings on PC1 and PC2, with PC2 being the least influenced. These features were crucial for detecting noisy ECG signals [23] but failed to capture most variation. However, given that the specificity of all SQIs typically declines as the window size increases, it was anticipated that these two SQIs might not exhibit the strongest discriminative power.

The MaxRR metric, which ranges from 0 to 1, does not require normalization. Lower-quality ECGs have values below 1, indicating interference from noisy peaks (Figure 5).

Figure 5 shows a minimal difference between unacceptable and acceptable ECG signals, indicating hosSQI’s relative stability across quality categories and lower sensitivity to factors defining the quality of ECGs from wearables.

As a result, achieving an Ac of 88%, precision of 92%, Se of 84%, and an F1 score of 87% with just eight ECG features underscores the efficacy and efficiency of the selected features in accurately classifying ECG segments.

4.3. Real-Life Conditions Analysis

As shown in Figure 6, the percentage of acceptable segments collected during the 8 h shifts for each condition highlights the importance of an ECG quality assessment algorithm. Interestingly, a higher proportion of low-quality ECG recordings (67.4% unacceptable) were observed during sedentary periods. This outcome is unexpected, as one might assume that resting conditions would produce more acceptable ECG segments due to reduced physical activity.

Being able to differentiate between acceptable and unacceptable ECG signals in these varying conditions is crucial for ensuring the reliability of data used to monitor the physiological state of firefighters in the field. Accurate ECG readings facilitate real-time assessments of their cardiac health, allowing for prompt interventions if any abnormal patterns arise. Additionally, this classification can be incorporated into automated monitoring systems to filter out poor-quality signals, leading to better decision-making and improved outcomes. By concentrating on high-quality data, these systems can offer valuable insights into the physical and mental well-being of firefighters, helping to mitigate fatigue-related accidents or health concerns.

5. Conclusions

This study investigated the SQA of ECG signals from wearable devices, targeting data collected in firefighters, utilizing machine learning. Continuous ECG monitoring is essential for diagnosing sporadic cardiac arrhythmias that are difficult to detect with short-duration ECGs. Firefighters face elevated cardiac risks due to adverse working conditions, making the implementation of wearable sensor systems crucial for monitoring their health.

The primary goal was to develop a supervised model using stratified 5-fold cross-validation to segment ECG signals into acceptable and unacceptable intervals, using a limited set of features. The ScientISST MOVE dataset, which contains ECG biosignals recorded with dry electrodes during various daily activities, provided a rich source of data for this study, offering 6468 segments of 10 s durations. The model’s performance was notably high, achieving an Ac of 88% and an F1 score of 87%, despite relying on only eight ECG features. This outcome highlights the potential of the selected features for the signal quality assessment of data collected in highly dynamic conditions.

Future work should address several limitations identified in this study, namely the following:

• Sample Size: Expanding the sample size could improve the ability to detect more nuanced differences, such as the impact of age and gender on the ECG signal quality.
• Dataset Diversity: Incorporating a more diverse dataset with both pathological and non-pathological ECG signals, as well as varying types of noise, is essential for developing more effective signal quality models.
• Model Hyperparameters: Further exploration of the hyperparameters for RFs, such as the maximum tree depth and the number of trees, could refine the model’s performance.
• Optimum Overlap: Finding the optimum overlap is crucial to reduce computational complexity.
• Overlap Percentages and Interval Lengths: Further exploration of a broader range of overlap percentages and interval lengths is necessary to optimize ECG signal segmentation.