Author Contributions
Conceptualization, S.S. and A.T.; Data curation, G.M.D., A.D., P.B., L.A. and F.D.P.C.; Investigation, S.S.; Methodology, S.S., G.M.D. and A.T.; Project administration, A.T.; Resources, G.M.D., G.M., A.D. and P.B.; Software, S.S.; Supervision, A.T.; Writing—original draft, S.S. and A.T.; Writing—review and editing, G.M.D., G.M., A.D., P.B. and A.T. All authors have read and agreed to the published version of the manuscript.
Figure 1.
Segmentation of the pre- and interictal states for the binary seizure forecasting task. The trace depicts 45 min of an EEG recording from the F7 channel of the Conegliano dataset during a seizure. Panels (A,B) illustrate a magnification of 5 s of recordings from 20 common channels of the inter- and preictal states, respectively.
Figure 1.
Segmentation of the pre- and interictal states for the binary seizure forecasting task. The trace depicts 45 min of an EEG recording from the F7 channel of the Conegliano dataset during a seizure. Panels (A,B) illustrate a magnification of 5 s of recordings from 20 common channels of the inter- and preictal states, respectively.
Figure 2.
The deep learning architecture contains six convolutional layers followed by batch normalization, pooling, and drop-out layers. Three dense layers are finally used to produce the output prediction.
Figure 2.
The deep learning architecture contains six convolutional layers followed by batch normalization, pooling, and drop-out layers. Three dense layers are finally used to produce the output prediction.
Figure 3.
Graphical representation of the two validation settings (RCV and LOO) considered in our experiments and the proposed calibration method, which exploits just one (Cal1) or two (Cal2) seizures of the target patient to fine-tune the forecasting model.
Figure 3.
Graphical representation of the two validation settings (RCV and LOO) considered in our experiments and the proposed calibration method, which exploits just one (Cal1) or two (Cal2) seizures of the target patient to fine-tune the forecasting model.
Figure 4.
Performance of the CNN model in the CHB-MIT dataset obtained with randomized cross-validation (RCV), leave-one-patient-out (LOO) validation and after Cal1 and Cal2 calibration. The violin plots illustrate the distribution of ACC, SEN, and SPE. The box plots with horizontal lines represent the interquartile range and the median.
Figure 4.
Performance of the CNN model in the CHB-MIT dataset obtained with randomized cross-validation (RCV), leave-one-patient-out (LOO) validation and after Cal1 and Cal2 calibration. The violin plots illustrate the distribution of ACC, SEN, and SPE. The box plots with horizontal lines represent the interquartile range and the median.
Figure 5.
The receiver operating characteristic (ROC) curves and the area under the curve (AUC) for LOO, Cal1, and Cal2 methods in the CHB-MIT and Conegliano datasets. y-axis and x-axis correspond to the true positive rate (sensitivity) and false positive rate (1—specificity), respectively.
Figure 5.
The receiver operating characteristic (ROC) curves and the area under the curve (AUC) for LOO, Cal1, and Cal2 methods in the CHB-MIT and Conegliano datasets. y-axis and x-axis correspond to the true positive rate (sensitivity) and false positive rate (1—specificity), respectively.
Figure 6.
Performance of the CNN model in the Conegliano dataset obtained with randomized cross-validation (RCV), leave-one-patient-out (LOO) validation and after Cal1 and Cal2 calibration. The violin plots illustrate the distribution of ACC, SEN, and SPE. The box plots with horizontal lines represent the interquartile range and the median.
Figure 6.
Performance of the CNN model in the Conegliano dataset obtained with randomized cross-validation (RCV), leave-one-patient-out (LOO) validation and after Cal1 and Cal2 calibration. The violin plots illustrate the distribution of ACC, SEN, and SPE. The box plots with horizontal lines represent the interquartile range and the median.
Figure 7.
Comparison of accuracy gains obtained by Cal1 and Cal2 with respect to the LOO baseline across all patients. Patients are sorted according to the maximum gain obtained by Cal2.
Figure 7.
Comparison of accuracy gains obtained by Cal1 and Cal2 with respect to the LOO baseline across all patients. Patients are sorted according to the maximum gain obtained by Cal2.
Figure 8.
Comparison between the CNN model (solid lines) and the XGBoost classifier (dotted lines) in terms of accuracy gain for the two calibration versions with respect to the LOO baseline. The blue lines refer to the CHB-MIT dataset while the green lines refer to the Conegliano dataset.
Figure 8.
Comparison between the CNN model (solid lines) and the XGBoost classifier (dotted lines) in terms of accuracy gain for the two calibration versions with respect to the LOO baseline. The blue lines refer to the CHB-MIT dataset while the green lines refer to the Conegliano dataset.
Table 1.
Performance of the CNN model for each patient in the CHB-MIT dataset. Each row corresponds to the ID, gender, and number of seizures per patient followed by ACC, SEN, and SPE values. Maximum values are highlighted in bold.
Table 1.
Performance of the CNN model for each patient in the CHB-MIT dataset. Each row corresponds to the ID, gender, and number of seizures per patient followed by ACC, SEN, and SPE values. Maximum values are highlighted in bold.
| RCV | LOO | Cal1 | Cal2 |
---|
ID | Gend. | No. Seizures | ACC (%) | SEN (%) | SPE (%) | ACC (%) | SEN (%) | SPE (%) | ACC (%) | SEN (%) | SPE (%) | ACC (%) | SEN (%) | SPE (%) |
---|
chb04 | m | 3 | 84.11 | 66.86 | 95.98 | 38.16 | 27.03 | 56.24 | 49.40 | 45.12 | 60.56 | 56.59 | 53.84 | 63.24 |
chb05 | f | 5 | 79.94 | 85.71 | 62.80 | 55.71 | 33.60 | 50.10 | 56.92 | 45.43 | 68.47 | 59.72 | 55.78 | 69.58 |
chb06 | f | 7 | 82.05 | 60.11 | 91.47 | 59.90 | 42.33 | 63.36 | 67.35 | 58.95 | 76.37 | 69.22 | 62.30 | 78.41 |
chb07 | f | 3 | 77.86 | 97.31 | 64.71 | 59.83 | 50.35 | 55.05 | 63.66 | 59.47 | 77.37 | 66.36 | 61.49 | 79.46 |
chb09 | f | 3 | 87.74 | 96.27 | 65.45 | 62.47 | 76.54 | 29.48 | 79.28 | 91.61 | 50.72 | 82.11 | 93.47 | 51.77 |
chb10 | m | 7 | 64.63 | 97.17 | 64.78 | 58.37 | 42.37 | 52.91 | 64.21 | 55.52 | 68.99 | 65.49 | 61.47 | 70.66 |
chb20 | f | 6 | 87.03 | 84.66 | 77.74 | 46.66 | 28.66 | 58.15 | 65.49 | 61.01 | 70.71 | 73.75 | 80.87 | 71.14 |
chb22 | f | 3 | 93.99 | 98.31 | 69.22 | 47.33 | 21.36 | 69.20 | 79.84 | 84.87 | 72.96 | 81.55 | 88.66 | 74.95 |
| | Average | 82.17 | 85.80 | 74.02 | 53.55 | 40.28 | 54.31 | 65.77 | 62.75 | 68.27 | 69.35 | 69.74 | 69.90 |
Table 2.
The average ACC, SEN, and SPE of the CNN model obtained from LOO, Cal1, and Cal2 in the CHB-MIT dataset, represented by the mean (%) ± std. The last two columns report the F-value and p-value from the ANOVA test.
Table 2.
The average ACC, SEN, and SPE of the CNN model obtained from LOO, Cal1, and Cal2 in the CHB-MIT dataset, represented by the mean (%) ± std. The last two columns report the F-value and p-value from the ANOVA test.
Metrics | LOO (Mean ± Std) | Cal1 (Mean ± Std) | Cal2 (Mean ± Std) | F-Value | p-Value |
---|
ACC | 53.55 ± 8.54 | 65.77 ± 10.26 | 69.35 ± 9.34 | 14.30 | <0.001 |
SEN | 40.28 ± 17.47 | 62.75 ± 16.96 | 69.74 ± 15.51 | 15.24 | <0.001 |
SPE | 54.31 ± 11.70 | 68.27 ± 8.82 | 69.90 ± 8.98 | 36.79 | <0.001 |
Table 3.
Performance of the CNN model for each patient in the Conegliano dataset. Each row corresponds to the ID, gender, and number of seizures per patient followed by ACC, SEN, and SPE values. Maximum values are highlighted in bold.
Table 3.
Performance of the CNN model for each patient in the Conegliano dataset. Each row corresponds to the ID, gender, and number of seizures per patient followed by ACC, SEN, and SPE values. Maximum values are highlighted in bold.
| RCV | LOO | Cal1 | Cal2 |
---|
ID | Gend. | No. Seizures | ACC (%) | SEN (%) | SPE (%) | ACC (%) | SEN (%) | SPE (%) | ACC (%) | SEN (%) | SPE (%) | ACC (%) | SEN (%) | SPE (%) |
---|
p1 | m | 4 | 97.12 | 98.00 | 96.09 | 35.40 | 38.96 | 31.46 | 57.58 | 68.72 | 50.55 | 59.36 | 76.69 | 51.10 |
p2 | m | 5 | 89.33 | 93.18 | 84.70 | 50.97 | 43.23 | 60.44 | 52.88 | 49.33 | 61.93 | 63.96 | 83.61 | 62.46 |
p3 | f | 5 | 88.82 | 88.82 | 88.83 | 50.71 | 52.97 | 48.73 | 68.24 | 54.87 | 84.53 | 70.61 | 60.68 | 88.21 |
p4 | f | 6 | 94.84 | 91.98 | 97.66 | 48.45 | 31.80 | 64.46 | 69.35 | 62.28 | 74.17 | 75.13 | 78.76 | 74.68 |
p5 | f | 4 | 99.18 | 99.86 | 97.93 | 53.91 | 50.63 | 58.98 | 80.15 | 82.35 | 77.02 | 84.96 | 93.10 | 79.54 |
p6 | f | 3 | 98.48 | 99.10 | 97.20 | 29.93 | 16.35 | 57.46 | 66.42 | 72.71 | 64.51 | 77.23 | 75.14 | 78.45 |
p7 | f | 7 | 97.46 | 95.68 | 99.60 | 50.47 | 51.04 | 49.80 | 56.11 | 64.82 | 51.89 | 62.41 | 81.19 | 61.74 |
p8 | m | 10 | 84.98 | 82.27 | 86.61 | 45.94 | 40.80 | 49.12 | 55.39 | 45.90 | 72.93 | 71.72 | 53.79 | 74.05 |
| | Average | 93.78 | 93.61 | 93.58 | 45.72 | 40.72 | 52.56 | 63.27 | 62.62 | 67.19 | 70.67 | 75.37 | 71.28 |
Table 4.
The average ACC, SEN, and SPE of the CNN model obtained from LOO, Cal1, and Cal2 in the Conegliano dataset, represented by the mean (%) ± std. The last two columns report the F-value and p-value from the ANOVA test.
Table 4.
The average ACC, SEN, and SPE of the CNN model obtained from LOO, Cal1, and Cal2 in the Conegliano dataset, represented by the mean (%) ± std. The last two columns report the F-value and p-value from the ANOVA test.
Metrics | LOO (Mean ± Std) | Cal1 (Mean ± Std) | Cal2 (Mean ± Std) | F-Value | p-Value |
---|
ACC | 45.72 ± 8.50 | 63.27 ± 9.34 | 70.67 ± 8.53 | 28.00 | <0.001 |
SEN | 40.72 ± 12.18 | 62.62 ± 12.24 | 75.37 ± 12.60 | 19.97 | <0.001 |
SPE | 52.56 ± 10.34 | 67.19 ± 12.10 | 71.28 ± 11.96 | 16.03 | <0.001 |
Table 5.
Tukey post hoc tests comparing the performance of calibrated and baseline models in the CHB-MIT and Conegliano datasets. Each row reports the p-value resulting from the comparison of ACC, SEN, and SPE metrics.
Table 5.
Tukey post hoc tests comparing the performance of calibrated and baseline models in the CHB-MIT and Conegliano datasets. Each row reports the p-value resulting from the comparison of ACC, SEN, and SPE metrics.
Dataset | Validation Methods | ACC | SEN | SPE |
---|
| LOO—Cal1 | <0.05 | <0.05 | <0.05 |
CHB-MIT | LOO—Cal2 | <0.01 | <0.01 | <0.05 |
| Cal1—Cal2 | 0.73 | 0.68 | 0.94 |
| LOO—Cal1 | <0.01 | <0.01 | <0.05 |
Conegliano | LOO—Cal2 | <0.01 | <0.01 | <0.05 |
| Cal1—Cal2 | 0.23 | 0.12 | 0.76 |
Table 6.
Average ACC, SEN, and SPE obtained by the XGBoost classifier in the CHB-MIT and Conegliano datasets (mean (%) ± std).
Table 6.
Average ACC, SEN, and SPE obtained by the XGBoost classifier in the CHB-MIT and Conegliano datasets (mean (%) ± std).
Dataset | Metrics | LOO | Cal1 | Cal2 |
---|
| ACC | 50.70 ± 7.83 | 56.26 ± 4.40 | 61.02 ± 5.81 |
CHB-MIT | SEN | 44.02 ± 10.38 | 52.89 ± 11.66 | 58.44 ± 10.47 |
| SPE | 60.95 ± 13.34 | 63.44 ± 12.65 | 66.84 ± 13.13 |
| ACC | 50.08 ± 6.71 | 58.49 ± 5.70 | 62.73 ± 4.71 |
Conegliano | SEN | 46.06 ± 19.54 | 69.22 ± 17.07 | 73.52 ± 16.34 |
| SPE | 50.21 ± 13.02 | 65.85 ± 11.44 | 70.68 ± 14.66 |
Table 7.
A comparison of different studies exploiting domain adaptation methods for cross-subject seizure forecasting in the CHB-MIT dataset.
Table 7.
A comparison of different studies exploiting domain adaptation methods for cross-subject seizure forecasting in the CHB-MIT dataset.
Authors | Year | Input Type | Classifier | SEN (%) | AUC |
---|
Peng et al. [29] | 2022 | spectrograms | Autoencoder | 73 | - |
Zhao et al. [54] | 2023 | raw signal | Gaussian mixture | 71 | 0.68 |
Liang et al. [52] | 2023 | raw signal | CNN | 89 | 0.85 |
Zhang et al. [53] | 2023 | spectrograms | Transformer | 80 | 0.81 |
Jemal et al. [55] | 2024 | raw signal | CNN | - | 0.75 |
This work | 2024 | raw signal | CNN | 70 | 0.85 |