*4.5. Signature Parameters*

Based on the determined correlation vectors for each pair of the transition (for*l* = 1... 2400) and the dictionary example*l*D = 1 ... 160, the values of features in the signature*S* were determined. For each example*l*, the signature contains 160 features. Values of two characteristic features for each category are presented in Figures 24 and 25.

**Figure 24.** Box plot of feature COR\_1\_A.

**Figure 25.** Box plot of feature COR\_9\_F.

The COR\_1\_A feature (Figure 24) can be used to distinguish between three categories: 1 (vacuum cleaner), 2 (slow juicer), and 13 (coffee machine). A majority of examples belonging to these categories have the value of the COR\_1\_A in the range between 0.97 and 1.0. Almost all observations of the remaining categories assume values of this feature in the range 0.85–0.97, so it is not suitable for distinguishing between them.

The COR\_9\_F (Figure 25) feature is characterized by high values for almost all examples. Only three vectors from category 3 (lamp with the "Osram" bulb) have COR\_9\_F values below 0.85. Observations for all other categories assume values of this feature in the range 0.85–1. Even though the COR\_9\_F feature was determined for the dictionary

examples belonging to the grinder category, the value distribution of this feature is similar for each category. Therefore it is not useful in most cases.

## *4.6. Classification Results*

This section presents results of three classifiers' operation for the available data partitioned using the K-fold cross validation (where *K* = 10).

#### 4.6.1. Neural Network

The application of ANN required selecting the optimal number of neurons in the hidden layer. For that purpose, the network was trained many times, with the number of neurons in the hidden layer ranging from 1 to 24. The classification error as a function of the number of neurons in the hidden layer is in Figure 26.

**Figure 26.** Classification error as a function of the number of neurons in the hidden layer.

The total classification error is minimal for 15 neurons in the hidden layer. Further increase of this parameter does not significantly affect the classification error.

The confusion matrix*C*NN and the classification accuracy*η<sup>n</sup>*EA, NN for 16 examined categories in the optimal ANN structure are presented in Table 4.

The overall classification accuracy was as follows:

$$
\eta\_{\rm ALL,NN} = \frac{1}{16} \sum\_{n\_{\rm EA}=0}^{15} \eta\_{n\_{\rm EA}\rm NN} = 78.6\%.\tag{9}
$$

The accuracy of at least 95% was obtained for nine appliances: 1 (vacuum cleaner), 2 (slow juicer), 7 (laptop), 8 (iron), 11 (kettle), 12 (jigsaw), 13 (coffee machine), 14 (air conditioner), and 15 (planer). Signatures for EAs 11–15 significantly differ from all others, which makes them easily identifiable (see columns 11–15). The lowest classification accuracy (24%) was achieved for category 9 (sharpener) and for category 3 (lamp with the "Osram" bulb) at the level of 28%. Total 56 (37%) examples of category 9 were incorrectly assigned to category 10. More than a half (51%) of examples of category 4 (lamp with the "Philips" bulb) were incorrectly assigned to category 5 (lamp with the "Omega" bulb). This is because categories 4 and 5 have similar signatures.

## 4.6.2. Decision Tree

None of the categories was faultlessly identified by DT (see Table 5). The best score of 97% was obtained for appliances 12 (jigsaw), 13 (coffee machine), 14 (air conditioner), and 15 (planer). For category 1 (vacuum cleaner), the accuracy was 95%. Categories 12 and 15 have very few examples from other incorrectly assigned categories. The worst classification result (29%) was obtained for category 3 (lamp with "Osram" bulb). There are also two groups of indistinguishable appliances. The first one consists of categories 0 (no EA), 3 (lamp with the "Osram" bulb), 6 (wall lamp with four "Lexman" bulbs), 9 (sharpener), and

10 (grinder). The second group consists of category 4 (lamp with bulb "Philips") and 5 (lamp with bulb "Omega"). The overall classification efficiency was as follows:

$$
\eta\_{\rm ALL, DT} = \frac{1}{16} \sum\_{n\_{\rm EA}=0}^{15} \eta\_{n\_{\rm EA}, DT} = 76.2\%. \tag{10}
$$


**Table 4.** Confusion matrix and the classification accuracy of the neural network for 15 neurons in the hidden layer.

**Table 5.** Confusion matrix and classification accuracy for the decision tree algorithm.


4.6.3. The kNN Algorithm

To identify the most significant predictors, the DT was first trained for all 2400 examples of transients. In the DT training process, 82 signature features were selected as the

most significant ones. Using selected features, a classification was made for each of the 10 cross-validation attempts. The confusion matrix*C*kNN for all cross-validation trials and the classification accuracy*η<sup>n</sup>*EA, kNN is presented in Table 6.


**Table 6.** Confusion matrix and classification accuracy for the k-nearest neighbors algorithm.

The k-nearest neighbors algorithm classified nine categories with an accuracy of at least 95%. These are: 1 (vacuum cleaner), 2 (slow juicer), 7 (laptop), 8 (iron), 11 (grinder), 12 (jigsaw), 13 (coffee machine), 14 (air conditioner), and 15 (planer). Categories 11, 12, 13, and 15 have been identified flawlessly. The worst classification result (39%) was obtained for category 10 (grinder). There are two groups of similar categories: (0, 3, 6, 9, 10) and (4, 5). The overall classification efficiency is as follows:

$$
\eta\_{\text{ALL, kN}} = \frac{1}{16} \sum\_{\eta\_{\text{EA}}=0}^{15} \eta\_{\eta\_{\text{EA}}, \text{ kN}} = 81.4\%. \tag{11}
$$
