Figure 1.
Outline of the proposed deep ensemble for HSI data analysis. The augmented models are obtained by modifying the base models by injecting the Gaussian noise into each model’s weights (note that more than one augmented model may be obtained from a single original model, as presented using red arrows). The final ensemble’s output can be a class label (for hyperspectral image data classification) or a vector of fractional abundances (for HU).
Figure 1.
Outline of the proposed deep ensemble for HSI data analysis. The augmented models are obtained by modifying the base models by injecting the Gaussian noise into each model’s weights (note that more than one augmented model may be obtained from a single original model, as presented using red arrows). The final ensemble’s output can be a class label (for hyperspectral image data classification) or a vector of fractional abundances (for HU).
Figure 2.
Visualization of the Indian Pines dataset (a), alongside the ground-truth segmentation (b).
Figure 2.
Visualization of the Indian Pines dataset (a), alongside the ground-truth segmentation (b).
Figure 3.
Visualization of the Salinas Valley dataset (a), alongside the ground-truth segmentation (b).
Figure 3.
Visualization of the Salinas Valley dataset (a), alongside the ground-truth segmentation (b).
Figure 4.
Visualization of the Pavia University dataset (a), and its ground-truth segmentation (b).
Figure 4.
Visualization of the Pavia University dataset (a), and its ground-truth segmentation (b).
Figure 5.
Visualization of the Houston dataset (a), alongside the ground-truth segmentation (b).
Figure 5.
Visualization of the Houston dataset (a), alongside the ground-truth segmentation (b).
Figure 6.
Visualization of the (a) Urban and (b) Jasper Ridge datasets for the hyperspectral unmixing.
Figure 6.
Visualization of the (a) Urban and (b) Jasper Ridge datasets for the hyperspectral unmixing.
Figure 7.
The results (overall RMSE and rmsAAD, averaged across Ur and JR) obtained using 1D-CNN and 3D-CNN (two upper plots), and the results obtained using 1D-DCAE and 3D-DCAE (two lower plots), as well as the scores elaborated using the ensembles and classical algorithms taken for comparison (LMM and R-SVR). For the supervised CNNs, we exploit the RF, DT, and SVR fusers, whereas for both supervised and unsupervised techniques we also utilize the ensemble that averages the predictions of base models (the mean aggregating variant). For each training set size, we build heterogeneous ensembles containing the 1D and 3D variants of the corresponding models (CNNs and DCAEs). Finally, we report the results obtained using the ensembles that include all models trained over all investigated sizes of training sets (the All variant for CNNs and DCAEs). We do not report the results for R-SVR for the largest training sets, as this method failed to train within the assumed time budget of 12 h.
Figure 7.
The results (overall RMSE and rmsAAD, averaged across Ur and JR) obtained using 1D-CNN and 3D-CNN (two upper plots), and the results obtained using 1D-DCAE and 3D-DCAE (two lower plots), as well as the scores elaborated using the ensembles and classical algorithms taken for comparison (LMM and R-SVR). For the supervised CNNs, we exploit the RF, DT, and SVR fusers, whereas for both supervised and unsupervised techniques we also utilize the ensemble that averages the predictions of base models (the mean aggregating variant). For each training set size, we build heterogeneous ensembles containing the 1D and 3D variants of the corresponding models (CNNs and DCAEs). Finally, we report the results obtained using the ensembles that include all models trained over all investigated sizes of training sets (the All variant for CNNs and DCAEs). We do not report the results for R-SVR for the largest training sets, as this method failed to train within the assumed time budget of 12 h.
Figure 8.
Training and prediction (test) times (in seconds, note the logarithmic scale—the test times are reported for the entire test sets) for all investigated training set sizes and algorithms (averaged across the datasets). We do not report the time results for R-SVR for the largest training sets, as this method failed to train within the assumed time budget of 12 h.
Figure 8.
Training and prediction (test) times (in seconds, note the logarithmic scale—the test times are reported for the entire test sets) for all investigated training set sizes and algorithms (averaged across the datasets). We do not report the time results for R-SVR for the largest training sets, as this method failed to train within the assumed time budget of 12 h.
Table 1.
The CNNs for hyperspectral image data classification. We present its hyperparameters, where k denotes the number of kernels, s is stride, denotes the number of hyperspectral bands, is the size of the input patch, and c is the number of classes in the considered dataset. The Conv, MP, and FC are the convolutional, max-pooling, and fully-connected layers, respectively, whereas ReLU is the rectified linear unit activation function.
Table 1.
The CNNs for hyperspectral image data classification. We present its hyperparameters, where k denotes the number of kernels, s is stride, denotes the number of hyperspectral bands, is the size of the input patch, and c is the number of classes in the considered dataset. The Conv, MP, and FC are the convolutional, max-pooling, and fully-connected layers, respectively, whereas ReLU is the rectified linear unit activation function.
Model | Layer | Parameters | Activation |
---|
1D-CNN | Conv1 | k: | ReLU |
| s: | |
Conv2 | k: | ReLU |
| s: | |
Conv3 | k: | ReLU |
| s: | |
Conv4 | k: | ReLU |
| s: | |
FC1 | | ReLU |
FC2 | | ReLU |
FC3 | | Softmax |
2.5D-CNN | Conv1 | | ReLU |
MP1 | | |
Conv2 | | ReLU |
Conv3 | | Softmax |
3D-CNN | Conv1 | | ReLU |
Conv2 | | ReLU |
Conv3 | | ReLU |
FC1 | | ReLU |
FC2 | | ReLU |
FC3 | | ReLU |
FC4 | | Softmax |
Table 2.
The CNNs for HU. We report the number of kernels, alongside their dimensions, and a denotes the number of endmembers.
Table 2.
The CNNs for HU. We report the number of kernels, alongside their dimensions, and a denotes the number of endmembers.
Variant | Layer | Parameters | Activation |
---|
1D-CNN | Conv1 | | ReLU |
Conv2 | | ReLU |
Conv3 | | ReLU |
Conv4 | | ReLU |
FC1 | | ReLU |
FC2 | | ReLU |
FC3 | | Softmax |
3D-CNN | Conv1 | | ReLU |
Conv2 | | ReLU |
Conv3 | | ReLU |
Conv4 | | ReLU |
FC1 | | ReLU |
FC2 | | ReLU |
FC3 | | Softmax |
Table 3.
The ground-truth color, and the number of examples for each class in Indian Pines.
Table 4.
The ground-truth color, and the number of examples for each class in Salinas Valley.
Table 5.
The ground-truth color and number of samples for each class in Pavia University.
Table 6.
The ground-truth color, and the number of examples for each class in the Houston dataset.
Table 6.
The ground-truth color, and the number of examples for each class in the Houston dataset.
Class | Description | GT Color | Number of Samples |
---|
1 | Healthy grass | ![Remotesensing 13 04133 i001]() | 39,196 |
2 | Stressed grass | ![Remotesensing 13 04133 i002]() | 130,008 |
3 | Artificial turf | ![Remotesensing 13 04133 i003]() | 2736 |
4 | Evergreen trees | ![Remotesensing 13 04133 i004]() | 54,322 |
5 | Deciduous trees | ![Remotesensing 13 04133 i005]() | 20,172 |
6 | Bare earth | ![Remotesensing 13 04133 i006]() | 18,064 |
7 | Water | ![Remotesensing 13 04133 i007]() | 1064 |
8 | Residential buildings | ![Remotesensing 13 04133 i008]() | 158,995 |
9 | Non-residential building | ![Remotesensing 13 04133 i009]() | 894,769 |
10 | Roads | ![Remotesensing 13 04133 i010]() | 183,283 |
11 | Sidewalks | ![Remotesensing 13 04133 i011]() | 136,035 |
12 | Crosswalks | ![Remotesensing 13 04133 i012]() | 6059 |
13 | Major thoroughfares | ![Remotesensing 13 04133 i013]() | 185,438 |
14 | Highways | ![Remotesensing 13 04133 i014]() | 39,438 |
15 | Railways | ![Remotesensing 13 04133 i015]() | 27,748 |
16 | Paved parking lots | ![Remotesensing 13 04133 i016]() | 45,932 |
17 | Unpaved parking lots | ![Remotesensing 13 04133 i026]() | 587 |
18 | Cars | ![Remotesensing 13 04133 i027]() | 26,289 |
19 | Trains | ![Remotesensing 13 04133 i028]() | 21,479 |
20 | Stadium seats | ![Remotesensing 13 04133 i029]() | 27,296 |
Total | | | 2,018,910 |
Table 7.
The results (BA, OA, and ) obtained using all investigated base models and various heterogeneous ensembles built with CNNs of different architectures (with all fusing schemes: Hard, RF, DT, and SVM) for all datasets: IP, SV, PU, H(A), and H(B) (averaged across all folds and executions). The best results for each row are boldfaced, the second best results are underlined, whereas the worst are grayed.
Table 7.
The results (BA, OA, and ) obtained using all investigated base models and various heterogeneous ensembles built with CNNs of different architectures (with all fusing schemes: Hard, RF, DT, and SVM) for all datasets: IP, SV, PU, H(A), and H(B) (averaged across all folds and executions). The best results for each row are boldfaced, the second best results are underlined, whereas the worst are grayed.
Model→ | Base Models | (1D, 2.5D, 3D) | (1D, 2.5D) | (1D, 3D) | (2.5D, 3D) |
---|
Metric↓ | 1D
| 2.5D | 3D | Hard | RF | DT | SVM | Hard | RF | DT | SVM | Hard | RF | DT | SVM | Hard | RF | DT | SVM |
IP | BA | 66.86 | 40.31 | 46.54 | 43.86 | 48.81 | 52.81 | 57.84 | 56.60 | 60.37 | 55.68 | 60.80 | 53.88 | 61.38 | 55.35 | 59.69 | 52.29 | 57.34 | 45.69 | 47.77 |
OA | 68.63 | 48.40 | 52.04 | 50.90 | 54.63 | 59.27 | 63.15 | 60.80 | 64.36 | 62.06 | 65.86 | 60.44 | 65.62 | 60.40 | 64.36 | 59.03 | 62.60 | 52.01 | 54.40 |
| 64.08 | 40.37 | 45.10 | 44.04 | 48.04 | 53.43 | 57.80 | 55.48 | 59.21 | 56.57 | 60.87 | 54.98 | 60.58 | 54.72 | 59.20 | 52.92 | 57.15 | 45.05 | 47.73 |
SV | BA | 81.39 | 69.98 | 79.31 | 72.03 | 76.88 | 77.52 | 79.18 | 79.67 | 83.99 | 74.82 | 76.72 | 73.75 | 79.93 | 81.04 | 82.96 | 79.79 | 82.08 | 75.46 | 75.97 |
OA | 78.01 | 69.60 | 76.46 | 72.93 | 75.45 | 76.86 | 76.70 | 78.41 | 81.16 | 74.08 | 75.20 | 75.10 | 77.00 | 79.07 | 79.61 | 77.03 | 78.85 | 74.87 | 74.72 |
| 75.17 | 65.74 | 73.43 | 69.34 | 72.26 | 73.80 | 73.66 | 75.50 | 78.71 | 70.74 | 71.97 | 71.73 | 74.02 | 76.36 | 76.95 | 74.04 | 76.09 | 71.59 | 71.41 |
PU | BA | 80.14 | 77.61 | 76.70 | 76.31 | 80.50 | 80.22 | 81.48 | 77.28 | 80.23 | 79.60 | 80.24 | 78.02 | 80.84 | 78.81 | 80.25 | 80.12 | 81.36 | 79.91 | 80.35 |
OA | 79.58 | 78.63 | 76.49 | 79.29 | 80.62 | 80.10 | 80.83 | 79.71 | 79.27 | 79.86 | 80.16 | 81.03 | 80.92 | 78.34 | 79.46 | 80.18 | 81.21 | 79.73 | 80.16 |
| 72.81 | 70.94 | 68.67 | 71.40 | 73.90 | 73.18 | 74.35 | 72.14 | 72.55 | 72.84 | 73.29 | 73.68 | 74.34 | 71.26 | 72.87 | 73.40 | 74.78 | 72.71 | 73.36 |
H(A) | BA | 47.67 | 38.76 | 38.75 | 39.69 | 38.56 | 37.49 | 37.15 | 35.44 | 38.37 | 38.24 | 36.22 | 35.45 | 40.17 | 37.60 | 35.20 | 40.61 | 39.61 | 38.47 | 39.03 |
OA | 59.85 | 52.27 | 51.40 | 54.48 | 52.00 | 49.91 | 51.83 | 53.57 | 51.38 | 50.80 | 51.78 | 53.85 | 53.04 | 50.24 | 50.66 | 55.70 | 53.17 | 50.80 | 52.39 |
| 47.11 | 38.73 | 37.91 | 40.57 | 38.57 | 36.37 | 38.15 | 37.35 | 37.84 | 37.35 | 38.02 | 37.89 | 39.76 | 36.34 | 36.62 | 41.61 | 39.77 | 37.30 | 38.95 |
H(B) | BA | 47.67 | 49.11 | 52.08 | 50.23 | 49.23 | 50.56 | 52.07 | 53.40 | 52.14 | 47.16 | 46.28 | 51.48 | 52.25 | 50.48 | 50.66 | 54.39 | 52.17 | 50.45 | 51.60 |
OA | 59.85 | 62.06 | 62.34 | 64.65 | 62.12 | 62.26 | 63.43 | 66.48 | 62.54 | 60.54 | 62.15 | 66.68 | 63.62 | 61.33 | 62.36 | 66.76 | 63.71 | 62.48 | 63.12 |
| 47.11 | 50.88 | 52.00 | 53.79 | 51.07 | 51.61 | 53.19 | 56.29 | 52.25 | 49.05 | 50.98 | 56.16 | 53.38 | 50.71 | 51.96 | 56.70 | 53.47 | 51.93 | 52.66 |
Table 8.
The results (BA, OA, and ) obtained using a single 1D-CNN model (1D), and using the ensembles with different numbers of augmented 1D-CNNs (# copies) with all fusing schemes (Hard, RF, DT, and SVM)—each ensemble always contains one original 1D-CNN model. We report the results for all datasets: IP, SV, PU, H(A), and H(B). Note that the augmented models were generated by injecting noise into the weights of the original model, and the number of such contaminated (augmented) copies may be freely updated. The best results for each row are boldfaced, the second best are underlined, whereas the worst are grayed.
Table 8.
The results (BA, OA, and ) obtained using a single 1D-CNN model (1D), and using the ensembles with different numbers of augmented 1D-CNNs (# copies) with all fusing schemes (Hard, RF, DT, and SVM)—each ensemble always contains one original 1D-CNN model. We report the results for all datasets: IP, SV, PU, H(A), and H(B). Note that the augmented models were generated by injecting noise into the weights of the original model, and the number of such contaminated (augmented) copies may be freely updated. The best results for each row are boldfaced, the second best are underlined, whereas the worst are grayed.
# Copies→ | 1 Copy | 2 Copies | 4 Copies | 8 Copies | 16 Copies |
---|
Metric↓ | 1D | Hard | RF | DT | SVM | Hard | RF | DT | SVM | Hard | RF | DT | SVM | Hard | RF | DT | SVM | Hard | RF | DT | SVM |
IP | BA | 66.86 | 63.48 | 68.58 | 66.08 | 67.48 | 64.51 | 68.50 | 66.67 | 67.61 | 62.59 | 68.41 | 65.63 | 67.41 | 60.84 | 68.47 | 67.05 | 66.54 | 60.67 | 68.66 | 67.37 | 66.90 |
OA | 68.63 | 64.72 | 70.39 | 66.68 | 69.81 | 65.67 | 70.42 | 67.28 | 69.67 | 63.90 | 70.50 | 67.16 | 69.69 | 63.06 | 70.37 | 67.12 | 69.58 | 63.14 | 70.38 | 67.02 | 69.65 |
| 64.08 | 59.69 | 66.31 | 61.94 | 65.41 | 60.55 | 66.36 | 62.55 | 65.24 | 58.48 | 66.46 | 62.40 | 65.25 | 57.62 | 66.32 | 62.44 | 65.12 | 57.73 | 66.34 | 62.31 | 65.19 |
SV | BA | 81.39 | 79.64 | 82.80 | 82.46 | 83.54 | 81.11 | 82.71 | 83.23 | 83.61 | 81.08 | 82.89 | 83.46 | 83.90 | 80.51 | 82.83 | 83.06 | 83.83 | 80.51 | 82.95 | 82.15 | 83.87 |
OA | 78.01 | 77.53 | 79.52 | 78.46 | 79.48 | 78.22 | 79.53 | 79.54 | 79.56 | 78.21 | 79.72 | 79.55 | 80.36 | 77.64 | 79.50 | 79.56 | 80.35 | 77.68 | 79.58 | 78.58 | 80.07 |
| 75.17 | 74.55 | 76.92 | 75.72 | 76.88 | 75.38 | 76.93 | 76.92 | 76.97 | 75.36 | 77.14 | 76.91 | 77.82 | 74.73 | 76.91 | 76.93 | 77.81 | 74.77 | 76.99 | 75.85 | 77.52 |
PU | BA | 80.14 | 79.61 | 79.82 | 79.67 | 80.71 | 80.01 | 79.82 | 80.04 | 80.81 | 80.05 | 79.81 | 79.91 | 80.92 | 80.11 | 79.62 | 79.82 | 80.93 | 80.19 | 79.69 | 79.74 | 80.91 |
OA | 79.58 | 79.43 | 78.68 | 78.57 | 80.42 | 79.10 | 78.74 | 79.03 | 80.45 | 79.02 | 78.72 | 78.25 | 80.45 | 78.93 | 78.70 | 78.36 | 80.46 | 78.94 | 78.70 | 78.11 | 80.40 |
| 72.81 | 72.51 | 71.85 | 71.64 | 73.88 | 72.29 | 71.92 | 72.21 | 73.92 | 72.21 | 71.89 | 71.33 | 73.93 | 72.12 | 71.84 | 71.29 | 73.95 | 72.15 | 71.86 | 71.12 | 73.88 |
H(A) | BA | 47.67 | 47.86 | 45.25 | 45.10 | 47.79 | 47.90 | 45.16 | 45.41 | 47.89 | 47.92 | 45.08 | 45.22 | 48.07 | 47.80 | 45.07 | 44.84 | 48.04 | 47.69 | 45.12 | 44.45 | 47.99 |
OA | 59.85 | 60.47 | 57.50 | 56.99 | 59.50 | 59.95 | 57.47 | 57.25 | 59.55 | 60.05 | 57.40 | 57.21 | 59.71 | 60.16 | 57.38 | 57.04 | 59.85 | 60.23 | 57.40 | 56.91 | 59.83 |
| 47.11 | 47.71 | 44.49 | 43.43 | 46.80 | 47.27 | 44.46 | 43.79 | 46.85 | 47.37 | 44.39 | 43.69 | 47.11 | 47.42 | 44.38 | 43.50 | 47.24 | 47.40 | 44.38 | 43.38 | 47.20 |
H(B) | BA | 47.67 | 55.01 | 54.38 | 54.60 | 56.95 | 56.06 | 54.22 | 54.44 | 57.03 | 56.15 | 54.18 | 54.14 | 57.26 | 56.20 | 54.14 | 54.40 | 57.41 | 56.29 | 54.16 | 54.68 | 57.52 |
OA | 59.85 | 65.93 | 64.47 | 64.35 | 66.51 | 65.73 | 64.36 | 64.32 | 66.69 | 65.69 | 64.25 | 64.32 | 66.80 | 66.03 | 64.24 | 64.30 | 66.90 | 66.05 | 64.15 | 64.33 | 66.97 |
| 47.11 | 56.08 | 54.77 | 54.26 | 56.99 | 56.07 | 54.67 | 54.20 | 57.10 | 56.10 | 54.57 | 54.22 | 57.25 | 56.47 | 54.57 | 54.18 | 57.40 | 56.46 | 54.49 | 54.25 | 57.58 |
Table 9.
The ranking obtained for all investigated HSI classification methods, averaged across all datasets (the best algorithm for a given set receives the ranking of 1, the second best: 2, and so on). We consider
to be the primary metric and, hence, sort the algorithms according to rankings obtained for
. We report the rankings for the (i) heterogeneus deep ensembles built with different CNNs (see the detailed results in
Table 7), (ii) augmented ensembles containing various numbers of noise-contaminated 1D-CNN (
Table 8), and (iii) all methods altogether.
Table 9.
The ranking obtained for all investigated HSI classification methods, averaged across all datasets (the best algorithm for a given set receives the ranking of 1, the second best: 2, and so on). We consider
to be the primary metric and, hence, sort the algorithms according to rankings obtained for
. We report the rankings for the (i) heterogeneus deep ensembles built with different CNNs (see the detailed results in
Table 7), (ii) augmented ensembles containing various numbers of noise-contaminated 1D-CNN (
Table 8), and (iii) all methods altogether.
Ensemble | Fuser | BA | OA | |
---|
Heterogeneous | RF (2.5D, 3D) | 4.2 | 4.6 | 4.2 |
RF (1D, 3D) | 3.4 | 5.4 | 5.0 |
Hard (2.5D, 3D) | 6.8 | 5.8 | 5.8 |
... | ... | ... | ... |
DT (1D, 2.5D) | 13.6 | 13.8 | 13.6 |
DT (2.5D, 3D) | 13.4 | 13.6 | 14.6 |
2.5D | 15.4 | 16.0 | 15.8 |
Augmented | SVM (4 copies) | 3.0 | 4.4 | 4.2 |
SVM (8 copies) | 4.4 | 4.2 | 4.2 |
SVM (16 copies) | 4.0 | 5.2 | 4.8 |
... | ... | ... | ... |
DT (4 copies) | 13.6 | 15 | 16.2 |
DT (1 copy) | 15.4 | 16.6 | 17 |
DT (16 copies) | 14.6 | 17.2 | 17.6 |
All | SVM (4 copies) | 3.6 | 5.6 | 5.0 |
SVM (8 copies) | 5.0 | 5.4 | 5.0 |
SVM (16 copies) | 4.6 | 6.4 | 5.8 |
... | ... | ... | ... |
DT (1D, 3D) | 33.6 | 30.6 | 30.2 |
3D | 31.4 | 34.2 | 33.4 |
2.5D | 35.4 | 35.2 | 35.8 |
Table 10.
Training time (in seconds) of all investigated base models, for all datasets.
Table 10.
Training time (in seconds) of all investigated base models, for all datasets.
Dataset | 1D-CNN | 2.5D-CNN | 3D-CNN |
---|
IP | 13.28 | 2.84 | 39.16 |
SV | 14.33 | 14.25 | 64.33 |
PU | 7.44 | 5.11 | 22.07 |
H(A) | 41.31 | 27.44 | 87.33 |
H(B) | 261.57 | 323.16 | 538.89 |
Table 11.
Training time (in seconds) of all investigated supervised fusers: RF, DT, SVM, for all datasets: IP, SV, PU, H(A) and H(B), obtained for the ensembles with different numbers of augmented 1D-CNNs (# copies). Each ensemble always contains one original 1D-CNN model.
Table 11.
Training time (in seconds) of all investigated supervised fusers: RF, DT, SVM, for all datasets: IP, SV, PU, H(A) and H(B), obtained for the ensembles with different numbers of augmented 1D-CNNs (# copies). Each ensemble always contains one original 1D-CNN model.
# Copies→ | 1 Copy | 2 Copies | 4 Copies | 8 Copies | 16 Copies |
---|
Dataset | RF | DT | SVM | RF | DT | SVM | RF | DT | SVM | RF | DT | SVM | RF | DT | SVM |
IP | 15.55 | 138.50 | 530.00 | 13.70 | 128.00 | 182.75 | 13.15 | 417.75 | 651.25 | 17.20 | 528.25 | 1214.00 | 28.15 | 179.00 | 1184.75 |
SV | 18.04 | 177.40 | 175.80 | 19.92 | 166.80 | 166.60 | 24.68 | 445.20 | 185.80 | 31.60 | 147.40 | 316.40 | 47.56 | 177.60 | 330.20 |
PU | 12.92 | 136.60 | 122.40 | 14.12 | 123.40 | 133.00 | 15.24 | 152.60 | 83.60 | 17.76 | 127.40 | 204.80 | 27.68 | 101.40 | 182.40 |
H(A) | 510.48 | 122.80 | 428.40 | 525.76 | 129.80 | 529.00 | 560.72 | 148.20 | 726.40 | 851.32 | 181.20 | 1104.00 | 897.68 | 346.60 | 1921.40 |
H(B) | 951.00 | 160.80 | 6164.60 | 1031.04 | 191.20 | 8969.40 | 1256.40 | 253.40 | 98,723.34 | 1509.88 | 358.60 | 108,325.67 | 2018.48 | 631.60 | 128,238.24 |
Table 12.
Training time (in seconds) of all investigated supervised fusers: RF, DT, SVM, for all datasets: IP, SV, PU, H(A) and H(B), obtained for the ensembles with different CNN architectures (various combinations of the 1D, 2.5D, and 3D models).
Table 12.
Training time (in seconds) of all investigated supervised fusers: RF, DT, SVM, for all datasets: IP, SV, PU, H(A) and H(B), obtained for the ensembles with different CNN architectures (various combinations of the 1D, 2.5D, and 3D models).
Ensemble→ | (1D, 2.5D, 3D) | (1D, 2.5D) | (1D, 3D) | (2.5D, 3D) |
---|
Dataset | RF | DT | SVM | RF | DT | SVM | RF | DT | SVM | RF | DT | SVM |
IP | 67.25 | 43.65 | 45.15 | 56.10 | 37.40 | 36.45 | 48.70 | 40.70 | 40.40 | 41.70 | 38.80 | 35.85 |
SV | 96.00 | 64.48 | 60.20 | 258.68 | 49.80 | 49.04 | 66.88 | 49.68 | 49.68 | 65.92 | 56.48 | 55.28 |
PU | 72.12 | 42.32 | 39.92 | 49.72 | 35.32 | 35.48 | 54.72 | 37.40 | 37.60 | 47.52 | 39.00 | 39.80 |
H(A) | 240.96 | 139.20 | 206.88 | 259.88 | 115.12 | 323.48 | 424.96 | 115.72 | 191.84 | 471.12 | 286.04 | 417.36 |
H(B) | 466.80 | 132.92 | 947.24 | 480.52 | 142.80 | 2142.32 | 621.80 | 177.84 | 1910.12 | 687.84 | 283.48 | 1380.80 |
Table 13.
Prediction time (in seconds, over the entire test set) of all investigated supervised fusers: RF, DT, SVM (together with the Hard voting), for all datasets: IP, SV, PU, H(A) and H(B), obtained for the ensembles with different numbers of augmented 1D-CNNs (# copies). Each ensemble always contains one original 1D-CNN model.
Table 13.
Prediction time (in seconds, over the entire test set) of all investigated supervised fusers: RF, DT, SVM (together with the Hard voting), for all datasets: IP, SV, PU, H(A) and H(B), obtained for the ensembles with different numbers of augmented 1D-CNNs (# copies). Each ensemble always contains one original 1D-CNN model.
# Copies→ | 1 Copy | 2 Copies | 4 Copies | 8 Copies | 16 Copies |
---|
Dataset | Hard | RF | DT | SVM | Hard | RF | DT | SVM | Hard | RF | DT | SVM | Hard | RF | DT | SVM | Hard | RF | DT | SVM |
IP | 1.17 | 1.16 | 1.17 | 1.46 | 2.14 | 1.30 | 1.29 | 1.66 | 4.53 | 1.61 | 1.59 | 2.12 | 6.88 | 2.20 | 2.18 | 3.07 | 9.37 | 3.39 | 3.37 | 4.96 |
SV | 3.59 | 7.00 | 3.47 | 5.62 | 4.62 | 8.13 | 4.44 | 7.16 | 6.82 | 10.04 | 6.40 | 10.38 | 11.34 | 14.38 | 10.37 | 16.61 | 21.59 | 22.34 | 18.23 | 29.37 |
PU | 2.13 | 2.37 | 3.20 | 2.91 | 2.58 | 2.73 | 2.41 | 2.75 | 3.48 | 3.51 | 3.09 | 3.55 | 5.48 | 5.00 | 4.41 | 5.07 | 10.37 | 7.95 | 6.73 | 9.04 |
H(A) | 55.65 | 468.63 | 17.81 | 322.48 | 61.62 | 478.32 | 24.63 | 421.66 | 78.88 | 508.47 | 40.25 | 616.14 | 111.01 | 790.50 | 68.99 | 988.35 | 177.04 | 816.69 | 132.52 | 1792.90 |
H(B) | 43.90 | 429.58 | 14.24 | 299.82 | 50.67 | 430.78 | 20.52 | 402.75 | 62.55 | 447.03 | 32.93 | 601.32 | 89.92 | 515.02 | 57.86 | 902.82 | 144.31 | 677.53 | 108.55 | 1582.83 |
Table 14.
Prediction time (in seconds, over the entire test set) of all investigated supervised fusers: RF, DT, SVM (together with the Hard voting), for all datasets: IP, SV, PU, H(A) and H(B), obtained for the ensembles with different CNN architectures (various combinations of the 1D, 2.5D, and 3D models).
Table 14.
Prediction time (in seconds, over the entire test set) of all investigated supervised fusers: RF, DT, SVM (together with the Hard voting), for all datasets: IP, SV, PU, H(A) and H(B), obtained for the ensembles with different CNN architectures (various combinations of the 1D, 2.5D, and 3D models).
| (1D, 2.5D, 3D) | (1D, 2.5D) | (1D, 3D) | (2.5D, 3D) |
---|
Dataset | Hard | RF | DT | SVM | Hard | RF | DT | SVM | Hard | RF | DT | SVM | Hard | RF | DT | SVM |
IP | 0.15 | 0.08 | <0.01 | 0.19 | 0.15 | 0.08 | <0.01 | 0.18 | 0.15 | 0.08 | <0.01 | 0.17 | 0.15 | 0.08 | <0.01 | 0.21 |
SV | 0.94 | 0.40 | 0.01 | 1.00 | 0.93 | 0.39 | <0.01 | 0.91 | 0.97 | 0.39 | <0.01 | 1.10 | 0.92 | 0.38 | 0.01 | 0.77 |
PU | 0.75 | 0.19 | <0.01 | 0.12 | 0.77 | 0.20 | <0.01 | 0.12 | 0.80 | 0.20 | <0.01 | 0.13 | 0.75 | 0.19 | <0.01 | 0.08 |
H(A) | 39.56 | 26.77 | 0.37 | 175.20 | 38.33 | 27.40 | 0.29 | 204.19 | 38.74 | 25.50 | 0.30 | 214.04 | 38.48 | 26.68 | 0.29 | 116.00 |
H(B) | 32.49 | 31.63 | 0.36 | 176.23 | 32.05 | 28.60 | 0.34 | 198.39 | 30.99 | 28.01 | 0.30 | 211.85 | 31.43 | 28.11 | 0.31 | 109.93 |
Table 15.
The DCAE architecture. We report the number of kernels, alongside their dimensions.
Table 15.
The DCAE architecture. We report the number of kernels, alongside their dimensions.
Variant | Layer | Parameters | Activation |
---|
1D-DCAE | Conv1 | | ReLU |
Conv2 | | ReLU |
Conv3 | | ReLU |
Conv4 | | ReLU |
Conv5 | | ReLU |
FC1 | | ReLU |
FC2 | | +Softmax |
FC3 | | ReLU |
3D-DCAE | Conv1 | | ReLU |
Conv2 | | ReLU |
Conv3 | | ReLU |
Conv4 | | ReLU |
FC1 | | ReLU |
FC2 | | +Softmax |
FC3 | | ReLU |
Table 16.
The results obtained using a single 1D-CNN model (1D) and using the ensembles with different numbers of augmented 1D-CNNs (# copies) with all fusers (Mean, RF, DT, and SVR); each ensemble always contains one original 1D-CNN. Note that the augmented models were generated by injecting noise into the weights of the original model, and the number of such contaminated (augmented) copies may be freely updated. We report the results for both datasets: Ur and JR. The best results for each row are bold, the second best are underlined, and the worst are grayed. We multiplied RMSE and rmsAAD by 100.
Table 16.
The results obtained using a single 1D-CNN model (1D) and using the ensembles with different numbers of augmented 1D-CNNs (# copies) with all fusers (Mean, RF, DT, and SVR); each ensemble always contains one original 1D-CNN. Note that the augmented models were generated by injecting noise into the weights of the original model, and the number of such contaminated (augmented) copies may be freely updated. We report the results for both datasets: Ur and JR. The best results for each row are bold, the second best are underlined, and the worst are grayed. We multiplied RMSE and rmsAAD by 100.
# Copies→ | 1 Copy | 2 Copies | 4 Copies | 8 Copies | 16 Copies |
---|
| Metric↓ | 1D | Mean | RF | DT | SVR | Mean | RF | DT | SVR | Mean | RF | DT | SVR | Mean | RF | DT | SVR | Mean | RF | DT | SVR |
---|
Ur | RMSE | 6.76 | 8.26 | 6.58 | 8.47 | 9.46 | 8.11 | 6.55 | 8.50 | 9.41 | 8.31 | 6.48 | 8.46 | 9.35 | 8.12 | 6.43 | 8.42 | 9.31 | 8.23 | 6.40 | 8.54 | 9.27 |
rmsAAD | 23.63 | 27.67 | 23.81 | 30.44 | 30.27 | 27.17 | 23.73 | 30.57 | 30.12 | 27.49 | 23.52 | 30.39 | 29.97 | 26.71 | 23.32 | 30.22 | 29.85 | 26.87 | 23.22 | 30.71 | 29.74 |
JR | RMSE | 11.95 | 12.58 | 9.79 | 11.72 | 12.20 | 12.45 | 9.56 | 11.72 | 12.02 | 12.73 | 9.46 | 11.57 | 11.97 | 12.78 | 9.40 | 11.56 | 11.96 | 12.92 | 9.25 | 11.66 | 11.88 |
rmsAAD | 29.88 | 31.46 | 25.00 | 30.11 | 30.00 | 31.07 | 24.49 | 30.22 | 29.52 | 31.73 | 24.18 | 29.79 | 29.36 | 31.63 | 24.07 | 29.84 | 29.33 | 31.87 | 23.72 | 30.18 | 29.17 |
Table 17.
Training and prediction (test) times (in seconds, the test times are reported for the entire test sets) for the investigated ensembles containing a given number of 1D-CNN copies (# copies), for all supervised fusers (RF, DT, and SVR, together with the mean fusion), and for a single 1D-CNN (1D).
Table 17.
Training and prediction (test) times (in seconds, the test times are reported for the entire test sets) for the investigated ensembles containing a given number of 1D-CNN copies (# copies), for all supervised fusers (RF, DT, and SVR, together with the mean fusion), and for a single 1D-CNN (1D).
# Copies→ | 1 Copy | 2 Copies | 4 Copies | 8 Copies | 16 Copies |
---|
| Time.↓ | 1D | Mean | RF | DT | SVR | Mean | RF | DT | SVR | Mean | RF | DT | SVR | Mean | RF | DT | SVR | Mean | RF | DT | SVR |
---|
Ur | Train | 6.21 | 4.20 | 5.89 | 5.58 | 9.87 | 4.56 | 5.25 | 4.79 | 4.86 | 5.09 | 6.12 | 5.99 | 5.63 | 20.29 | 8.43 | 8.21 | 7.72 | 12.19 | 16.83 | 14.44 | 13.95 |
Test | 0.39 | 0.37 | 0.56 | 0.26 | 0.55 | 0.57 | 0.69 | 0.40 | 0.75 | 0.97 | 0.94 | 0.64 | 1.17 | 1.98 | 1.49 | 1.18 | 2.03 | 5.06 | 2.48 | 2.27 | 3.81 |
JR | Train | 4.01 | 3.60 | 3.94 | 4.52 | 4.95 | 3.89 | 4.65 | 4.80 | 5.61 | 4.57 | 5.65 | 4.97 | 5.82 | 6.16 | 7.13 | 7.95 | 6.98 | 10.03 | 13.66 | 13.22 | 12.99 |
Test | 0.09 | 0.15 | 0.06 | 0.04 | 0.04 | 0.22 | 0.07 | 0.05 | 0.05 | 0.40 | 0.08 | 0.06 | 0.08 | 0.90 | 0.12 | 0.10 | 0.11 | 2.93 | 0.19 | 0.16 | 0.17 |