*3.3. Data Classification*

For the same purpose as [20], but only with the accelerometer data, this study aimed to recognize the five proposed ADLs being used, based on the datasets presented in Figure 7. The granularity of the features included varies between the datasets 1–5; i.e., the dataset 5 contains all inputs of datasets 1 to 5.

**Figure 7.** Datasets created for the analysis and recognition of the different ADLs.

For this purpose, we used three different implementations with distinct configurations using free software available online. The application of the MLP method takes into account the same settings, but two different implementations were performed using the Neuroph [17] and Encog [18] frameworks. Additionally, we used the DeepLearning4j framework for the application of a DNN method [19]. These are Java-based frameworks that allow for the implementation of machine learning methods with the adaptation to our data. All configurations of the frameworks implemented the sigmoid function as the activation function, a maximum of 4 x 10<sup>6</sup> iterations and backpropagation [69]. However, the learning rates applied in the MLP implementations and the DNN method are different; the value was 0.6 for MLP implementations and 0.1 for the DNN method. The MLP implementations also included the momentum value equal to 0.4. Regarding the numbers of hidden layers, the MLP methods did not include hidden layers, but the DNN method implemented three hidden layers. The DNN method also included the Xavier function [70] as a weight/initialisation function, a seed value equal to 6, and L2 regularization [71]. After different tests and adjustments, we verified that these parameters

#### *Electronics* **2020**, *9*, 509

reported more consistent results with the data acquired than others, suggesting its implementation in the developed method.

Additionally, the data classification was tested with normalized and non-normalized data, implemented the min-max normalization for the implementations of the MLP method, and the normalization with mean and standard deviation for the implementation of the DNN method.

#### **4. Results and Discussion**

As the different implementations reported the existence of overfitting during the creation of the different ANNs, the early-stop training technique was implemented, stopping the training at a limit of 4 × 10<sup>6</sup> iterations. Thus, the results reported are presented in Figures 8 and 9 for non-normalized and normalized data, respectively.

**Figure 8.** Results obtained with the MLP method implemented using non-normalized data with Neuroph and Encog frameworks, and the DNN method implemented with the DeepLearning4j framework (horizontal axis) for the different datasets (series), obtaining the accuracies in percentages (vertical axis).

**Figure 9.** Results obtained with the MLP method implemented using normalized data with Neuroph and Encog frameworks, and the DNN method implemented with the DeepLearning4j framework (horizontal axis) for the different datasets (series), obtaining the accuracies in percentages (vertical axis).

After the implementation with the Neuroph framework, the results obtained had very low accuracies with normalized (between 20% and 30%) and non-normalized (between 20% and 40%) data. Following the implementation with the Encog framework, the results obtained had a very low accuracy (between 20% and 40%) with data without normalization, wherein, as excepted, the neural networks trained with the dataset 5 reported a certainty around 75%. When the data were normalized, the accuracy of the implemented method was always between 10% and 40%.

Next, for the implementation with the DeepLearning4j framework, the results obtained are higher than 70%, but, for data without normalization, the results reported with the dataset 5 have an accuracy lower than 30%, and for the normalized data, the results decrease with a reduced number of features—dataset 5 reported the best results.

There are two types of normalization implemented with the data acquired, including the one based on mean and standard deviation and the other one based on min-max. The accuracy reported for non-normalized data is better than the accuracy reported for data with min-max normalization. However, the results with all defined datasets increase with the application of L2 regularization and normalization with mean and standard deviation.

Table 2 shows the maximum accuracies obtained with the MLP method with Neuroph and Encog frameworks and the DNN method with the DeepLearning4j framework. The DeepLearning4j framework reported the best accuracy, and the results obtained by Neuroph and Encog frameworks are not satisfactory.


**Table 2.** Best accuracies obtained with the different frameworks and datasets.

Analyzing the results presented in Table 2, Neuroph framework always reported bad results with an accuracy of 32.02% using dataset 5 using non-normalized data, and an accuracy of 24.03% with dataset 3 using normalized data. Among the frameworks used in this study, the Neuroph framework reported the worst results, because its architecture is not adapted for this type of data, or because it needs a large number of samples for the training of the ANN. The Neuroph framework reported better results with a large number of inputs for the ANN.

The use of the Encog framework slightly improved the results obtained with normalized data, reporting an accuracy of 37.07% using the dataset 2. However, Encog framework reported a high accuracy with the use of non-normalized data (74.45%). In contrast with the Neuroph framework, it was verified that the best accuracies were attained by the implementations with a smaller number of inputs.

The major problem of the implementation of DeepLearning4j framework is the resource consumption, where the performance is affected. However, the performance is only bad in the training phase. The final implementation the ANN provides reliable results after being trained. DeepLearning4j always reported high accuracy in the results with a large number of inputs—the results obtained were 80.35% accurate with non-normalized data, and 85.89% with normalized data.

The results recommend the DNN method with all features extracted from the acquired data as the most reliable method for the identification of ADLs. However, before its implementation, the data should be normalized with the mean and standard deviation method, and the L2 regularization method should be applied. Based on the tests performed with the acquired data, the results obtained are always higher than those reported other ways. The results obtained have a *precision* value of 86.21%, a *recall* value of 85.89%, and an *F1 score* value of 86.05%.

In addition to the analysis, the confusion matrixes for the different frameworks were made, and are presented in Tables 3–8. By analyzing Table 3, it is possible to verify that the number of true positive values in recognition of walking upstairs, walking downstairs, and standing, is meager, proving a high number of false negatives and true negatives using the MLP method with the Neuroph framework based on non-normalized data. Next, Table 4 shows that the number of true positive values in recognition of all ADLs is meager, verifying a high number of false negatives using the MLP method with the Neuroph framework based on normalized data.

Following the analysis of Table 5, it was verified that only running is recognized by the MLP method with the Encog framework based on non-normalized data, presenting a high number of false negative values. In contrast, based on the implementation of the MLP method with the Encog framework based on normalized data, walking is always correctly recognized with 2000 true positive values, but it has 7999 false negative values. The high number of false negative values is also verified in the other ADLs, and the true negative and false positive values are too high.

Based on the use of the DNN method with the DeepLearning4j framework based on non-normalized data, the number of true negatives is only low in recognition of standing activity, reporting a high number of false positive values. However, the standing activity also reported a high number of true positive values, while the other ADLs reported high false negative values. Finally, with the use of the DNN method with the DeepLearning4j framework based on normalized data, the true positive and true negative values are high in all ADLs recognized.


**Table 3.** Confusion matrix of the results obtained with non-normalized data by the implementation of the MLP method with the Neuroph framework.

**Table 4.** Confusion matrix of the results obtained with normalized data by the implementation of the MLP method with the Neuroph framework.


**Table 5.** Confusion matrix of the results obtained with non-normalized data by the implementation of MLP method with Encog framework.



**Table 6.** Confusion matrix of the results obtained with normalized data by the implementation of the MLP method with the Encog framework.

**Table 7.** Confusion matrix of the results obtained with non-normalized data by the implementation of the DNN method with the DeepLearning4j framework.


**Table 8.** Confusion matrix of the results obtained with normalized data by the implementation of the DNN method with the DeepLearning4j framework.


This paper highlights the results obtained with different datasets using only the accelerometer data for the creation of a part of the method for the automatic recognition of several ADLs, including running, walking, walking upstairs and downstairs, and standing. The study also compares the results obtained with different types of ANNs, requiring low processing for the correct implementation in mobile devices.

The low accuracies verified with Neuroph and Encog frameworks are related to the fact that the ANNs created are probably overfitted. The possible solutions may be the acquisition of more data, the application of L2 regularization, the implementation of dropout regularization, the early stopping of the training, the use of the batch normalization, or the use of a minor number of features in the ANN. The DNN method with L2 regularization and normalized data reported the best results. The influence of the amount of the maximum iterations is not substantial, but, in some cases, it increases the accuracy of the ANN.

During the data acquisition, several constraints may exist, collecting noised values of sensors' data. Commonly, the accelerometer is available in all mobile devices, and the implementation of the system architecture for the recognition of ADLs and its environments can be possible with all devices in the market. However, these are multitasking devices, and sometimes the data cannot be collected or is incorrectly collected, providing low accuracy on the recognition of the ADL. Another example consists of the positioning of the mobile device because the data is not correctly acquired during a call. Memory and power processing are profoundly affected by the performance of different tasks at the same time.

The main focus of this research was to explore the use of the accelerometer sensor for ADLs recognition. We found that the accuracy obtained is in line with the previous results in the literature [20]. This study reports an accuracy of 85.89% in the recognition of five ADLs. Furthermore, using the

DNN method, according to Table 2, the results obtained with the implementation of our methods are not directly comparable, because the datasets and source code of the implementation used by other authors are not publicly available. A comparison would be essential to proving the reliability of our method. Thus, considering the average of the accuracies reported by ANNs and their variants shared in the literature, the results (92% ± 6.55%) present better accuracies than those obtained in this study. However, taking into account only the average of the accuracies reported by the projects that identified more than one ADL, the results reported by other studies (90% ± 6.60%) are slightly equivalent to those published by our research. Finally, considering only the studies that recognized five or more ADLs, the results reported by these studies (90% ± 6.63%) are equivalent to the results obtained with this work.

In conclusion, the accuracy of the ADLs recognition depends on several variables, including the conditions for data acquisition, conditions for data processing, and the use of lightweight methods (local processing) or server-side processing [72]. As presented in [72], it may cause failures on the data acquisition, collect incorrect data, or claim the nonexistence of data in some instances, causing improper recognition of ADL. To avoid some effects of inaccurate data, we implemented data cleaning methods, and data imputation methods may be useful for reducing the impacts of unavailable data. The main possible problems are related to the incorrect or nonexistent recognition of ADLs performed.

The main limitations of this study are related to the use of mobile devices for data acquisition. On the one hand, there is a lack of scientific evidence and research on the definition of the best position at which the mobile device must be located. On the other hand, other constraints during the data acquisition are related to the frequency of the data acquisition because it depends on the different processes running in the mobile device. During the experimental phase, the mobile application developed for the data acquisition writes the data in text files; the latency to write in the text files also influences the data acquisition and processing. However, the use of local processing and lightweight methods reduces the lag of the connection with the network, but the different methods must always be optimized.

Taking into account the results obtained in [43], the number of ADLs recognized, the number of records for each ADL, and the features extracted are different in our study. Consequently, the accuracy obtained in our research with the DNN method is higher than the results reported by the authors of [43]. We expect that in similar conditions of study [43], we obtain the same or better results. Nevertheless, it will be impossible to test, as the authors [43] did not make their data publicly available.
