A Granular Computing Classifier for Human Activity with Smartphones

Abstract

Recently, smart home devices have been widely used to assist and facilitate the lives of human beings. Human activity recognition (HAR) aims to identify human activities using sensors in smartphones. Therefore, it can be employed in many applications, such as remote health monitoring for disabled and elderly people. This paper proposes a granular computing-based approach to classifying human activities using wearable sensing devices. The approach has two main phases: feature selection and classification. In the feature selection phase, the approach attempts to remove redundant and irrelevant attributes. At the same time, the classification phase makes use of granular computing concepts to build the granules and find the relationships between granules at different levels. To evaluate the approach, we applied the dataset to five famous machine learning models. For the comparative evaluation, we also tested other well-known machine learning methods. The experimental results presented in this paper show that the approach outperformed common traditional classifiers in terms of classification precision recall, f-measure, and MCC for most recognized human activities by approximately 97.3%, 94%, 95.5%, and 94.8%, respectively. However, in terms of processing time, it performs comparably.

1. Introduction

Human activity recognition has become an important research area in the machine learning literature. Due to the development of games, TV, and other technologies that depend on human motions, many researchers are looking for the best classifier that can efficiently determine human motions while outperforming other classifiers in terms of performance accuracy [1]. Human activity recognition makes use of machine learning techniques to determine common activities in real life, such as sitting, walking, and standing. Accordingly, human activity recognition can be used to identify personal relationships among people by collecting data from multiple objects at the same place. It can be applied in various fields such as security, surveillance, games, and remote monitoring [1]. A human activity may be a simple or complex movement. Examples of simple movements include walking, lying, running, standing, and setting. However, complex movements contain walking downstairs and walking upstairs. Furthermore, a complex movement can be decomposed into several simple movements [2].

There are several devices that are used to capture human motion, such as cameras or sensors. Cameras can be employed to record the human activity videos, and then computer vision techniques are utilized to answer two questions: “What is the action?” and “Where is the action in the video?” In recent years, researchers have focused on “dense sensing”, where many sensors are distributed in the environment to capture human activities [2].

Recently, various methods have been introduced to solve the human activity recognition problem with efficient accuracy. The study by Ahmed et al. (2020) improved human activity recognition based on sensor data. This model employs a hybrid feature selection model using sequential floating forward search (SFFS) to select the most representative features. It makes use of a multiclass support vector machine (SVM) to classify the observations into its labels. This model achieved higher classification accuracy than traditional machine learning classifiers [3].

The study by Mohamed et al. (2018) presented a technique for recognizing human activities based on a multi-label classification of various accelerometer sensor positions. In order to achieve high accuracy, this technique focuses on determining the optimal sensor placement in relation to the performed activity.It can recognize six different types of activities. Furthermore, it compares among traditional multi-class and multi-label classification methods such as rotation forest [4], SVM [5], and Iterative Dichotomizer 3 (ID3) [6]. However, this model relies on traditional machine learning classifiers, so it searches for the one that gives the highest classification accuracy.

Recently, (Burns and Whyne 2020) suggested a novel HAR approach aimed at personalizing human activity recognition based on deep triplet embedding. The subject triplet selection makes use of triplet neural networks for personalized activity recognition. Therefore, it improves the performance of the training process. However, the latency time increased as the size of the input data increased [7].

Also, (Archana and Hareesh 2021) suggested that Resnet and 3D CNN recognize human activities in real-time. And the accuracy of recognition can be improved by combining ResNet with 3D CNN. As a result, a technique for identifying, tracking, and detecting real-time human movements has been created [8]. The study by Giovanni et al. (2013) describes a system for identifying and reporting unusual behaviors. The suggested approach uses the situation awareness paradigm to identify unusual behavior. The architecture adds temporal information, including duration and frequency, to the recognition process [9].

Giovanni and Antonio’s (2017) offer a method for recognizing that Alzheimer’s disease is the most prevalent kind of dementia. Patients with this type of dementia may exhibit symptoms like restless sleep, muscle rigidity, or other standard Alzheimer’s movement anomalies. They have concentrated on the alterations connected to sleep problems. The fact that sleep disorders are not well-known makes it challenging to create software that can recognize them. Using deep learning algorithms, they were able to solve the issue of automatically identifying sleep abnormalities in Alzheimer’s disease patients [10].

Babiker et al. (2017) show the development of an intelligent human activity identification system. Each level of the suggested system employed a variety of digital image processing techniques, including background subtraction, binarization, and morphological operations. Based on the human activity features database that was taken from the frame sequences, a strong neural network was constructed. A multi-layer feed forward perceptron network was used to categorize the dataset’s activities [11].

This article aims to determine the set of human activities based on a predefined, finite set of activities. Well-known classification models could be used to solve this problem. Therefore, the process of recognition consists of two phases: the training and testing phases. In the training phase, well-known samples of human activities are used to generate a supervised model. This model can differentiate among the characteristics of human activities. However, the testing phase evaluates the performance of the established model by taking activity samples without their labels. Moreover, the models are evaluated by comparing the true state of nature to the expected activities [12]. This paper attempted to answer the following questions: “What information is useful in determining human action?” (i.e., a feature selection problem), “What is the action?” (i.e., a recognition problem), and “What is the response time?” (i.e., a problem of time complexity).

The rest of the paper is organized as follows: Section 2 presents basic concepts, while Section 3 provides the approach in detail. However, the experimental results and discussion are introduced in Section 4. Finally, Section 5 concludes this work.

2. Basic Concepts

In this section, a literature review for feature selection methods is presented. Also, the rule induction granular computing is introduced.

2.1. Feature Selection

The recent explosion of dataset size, in both number of observations and dimensions, has pushed towards the usage of data reduction techniques. Moreover, increasing dataset size might sometimes produce worse performances in data analytics applications. Furthermore, data reduction can improve storage efficiency and reduce costs. Data reduction can be achieved either by decreasing the number of objects or by decreasing the number of features. Therefore, data reduction is considered a significant preprocessing task for big data analytics techniques [13].

Feature reduction techniques can be divided into two main groups: feature construction and feature selection. Feature construction techniques transform the original features in high-dimensional space into new low-dimensional features. The newly generated features are usually linear or nonlinear functions of the original features, e.g., principal component analysis (PCA) [14], singular value decomposition (SVD) [14], and linear discriminant analysis (LDA) [14].

On the other hand, feature selection aims at removing a group of irrelevant and redundant features, attributes, or variables [15]. As shown in Figure 1, these methods may be classified according to the type of learning algorithms into three main approaches: supervised, unsupervised, and semi-supervised. Unsupervised feature selection techniques do not require labeled data. These techniques are employed before supervised machine learning algorithms such as clustering. They focus on finding the set of features that maximize clustering quality measures [16,17]. When some subsets of data are labeled but others are not, semi-supervised feature selection techniques are used [18]. Supervised feature selection methods are used in conjunction with supervised machine learning algorithms such as classification or regression, where the decision or class variable is present. These techniques attempt to select a group of features that maximize the accuracy of the predicted decision variable using predefined supervised techniques [14,19]. They require the availability of labeled data. Based on the selection methodologies, supervised feature selection algorithms are further classified into three types: wrapper, filter, and hybrid [20].

Wrapper feature selection methods make use of optimization algorithms to choose the set of features that improve the classification accuracy of a predefined learning algorithm [21]. However, filter feature selection algorithms choose features based on their intrinsic characteristics [22]. On the other hand, hybrid feature selection methods combine the advantages of both filter and wrapper feature selection algorithms. They often consist of two phases. In the first phase, they employ either a filter or a wrapper technique as the initial step for minimizing a set of features. while another type of feature selection algorithm is used in the next phase to select the final set of attributes [23]. Filter feature selection methods are widely used in the literature because they are usually fast and scalable techniques [23]. This paper is concerned with supervised feature selection techniques due to the availability of class labels for human activities. It focuses on filter-based feature selection techniques to choose the list of the most relevant features for recognizing human activities.

There are three types of filter feature selection techniques: statistical and informational, bioinspired, and spectral based. The first group makes use of statistical and/or information measures such as variance, entropy, and mutual information among features. Bio-inspired techniques employ bio-inspired stochastic optimization search strategies for choosing a good subset of features based on some quality criteria. However, the third group makes use of spectral analysis to find the best representative features [24].

Finally, statistical feature and information-based selection algorithms can be further classified into two subcategories: univariate and multivariate techniques. Univariate feature selection algorithms detect redundant features by ranking them individually, for example, based on their association with a class variable or information gain. However, multivariate feature selection techniques handle features jointly rather than individually. Multivariate methods are used to detect redundant and irrelevant features [25].

Nowadays, feature selection plays an important role in machine learning projects because a good feature selection algorithm influences overfitting, improves accuracy, and minimizes training time [15]. Therefore, this paper focuses on multivariate filter based feature selection because the set of human activities has class labels and the other features have numerical values. Each record in the dataset has a triaxial acceleration from the accelerometer (total acceleration) and the estimated body acceleration, a triaxial angular velocity from the gyroscope, a 561-feature vector with time and frequency domain variables, and its activity label.

2.2. Granular Computing

Granulation aims at grouping elements based on their similarity, functionality, proximity, or indistinguishability. Granular Computing (GrC) formally represents a granule as a subset of a universal set [26]. In machine learning, the classification problem can be seen as a model of inductive learning, which exploits the training sets to build a model that provides an efficient representation for all testing sets. Granular computing employs the partitioning process by learning a classifier. It employs a rule induction algorithm that is applied using supervised machine learning techniques [27].

Computing the granules can be regarded as knowledge processing. Granules can be defined as the basic unit of knowledge. Each granule can be perceived by a human’s knowledge processing. Therefore, unknown concepts (a subset of the universe dataset) can be approximated by known grains according to the axioms of approximation [27].

The rule induction technique is used in granular computing to emulate the supervised machine learning methods used to classify the dataset. In rule induction, a set of rules (r1, r2, …, rm) is expressed to map the input space to the output space. Each rule consists of a logical implication, and the results become valid if the prediction is satisfied [28,29]. AQ-family [30] and CN2 [31] are examples of traditional rule induction methods. In GrC, every rule, ri, is associated with a certain subset of instances; it might be considered a granule from the GrC point of view [27].

Definition 1.

In the two-dimensional plane space of the information system S = (U, R), the elementary granulation in each granulation is designated by a point in the plane, known as the granular node. For elementary granulations within the same granulation, there is no line segment connecting; however, for elementary granulations within different granulations, a line segment connecting in the plane space denotes that the two granulations have the intersection set. This line segment is referred to as the granulation side, and the intersection set on the granulation side denotes the granulation side’s weight value. As a result, a granular network in two dimensions is created [32].

Definition 2.

A directed weighted graph can be used to represent the entire granular network of a decision table since each granulation side is represented by a directed line segment from left to right, as shown in Figure 2 [32].

The following stages are used to build granular computing network (GrC) for the dataset S = (xi, yi) | I = 1, 2, …, n in N-dimensional space. First, the point in S is symbolized by the indivisible atomic grains. Second, based on the set’s view, a distance between two granules is suggested. Thirdly, the union process is jointly determined by distance and granularity. Finally, the class of test data is predicted using the granule set, which is obtained at this point.

2.2.1. Granularity and Granule Representation

In fact, the union granule and the meet granule are related to the shapes of the granules; the distance between two granules is not easily calculated; and the shapes of the granules are uneven. To explore granular computing, the granule is represented as regular shapes such as a hyperdiamond, a hypersphere, hypercube, and a hyperbox. The hyperdiamond granules represented as a vector include the center of the hyperdiamond and half of its diagonal length. The hypersphere granules represented as vectors include the center and radius of the hypersphere. A vector that includes the hypercube’s center and one-half of its side length is used to represent a hypercube granule. The hyperbox granule is represented as a vector with vectors induced at both the beginning and end points.

In this paper, the granule computing network calculates the distance between two hypesphere granules (1) using the Euclidean distance (2) between the two centers of each granule.

$$g_r\left(G\right)=r$$

(1)

where r is the radius of hypersphere.

$$g_r={\parallel x-y\parallel }_2$$

(2)

2.2.2. Operators between Granules

Any point is regarded as an indivisible atomic granule; the union process is required to generate larger granules than atomic granules. The decomposition process is important for breaking up bigger grains into smaller ones, since the whole universe is a granule with the highest degree of granularity. The union hypersphere granule is (3) for two sphere granules, $G_1=(C_1,r_1)$ and $G_2=(C_2,r_2)$.

$$G=G_1\vee G_2$$

(3)

3. Methods and Materials

In this section, the model of human activity recognition is briefly described in a detailed procedure. The model is diagrammatically shown in Figure 3. The model consists of two major phases: the first phase targets selecting the optimal set of attributes from the dataset, and the second phase presents the model, which aims at classifying human motion based on the concept of a granular computing classifier. Moreover, the quality of the raw dataset is low. Therefore, the approach performs the data preprocessing steps such as missing value imputation, feature normalization, and data encoding.

3.1. Feature Selection

In this phase, the dataset is split into two subsets: the first subset contains 70% of the whole dataset for training, and the second subset includes 30% for testing. The first subset is used to generate the optimal set of attributes. The feature selection method called fast correlation based filter (FCBF), FCBF is a filter selection for high dimensional data and can remove a large number of features that are redundant. FCBF is used to reduce the attributes based on the pairwise correlation between each attribute and the decision attribute [33], as shown in Figure 4.

3.2. Granular Computing Classifier

The granular computing classifier has two major parts: a granule builder and a granular computing network.

3.2.1. Granules Builder

The granules builder method describes the number of objects in granules that are similar in a specific formula. These objects are called “basic granules” [27]. The fundamental idea is that the pair (a, m) represents the value of the decision attribute and the set of objects that are correlated with this value. Each pair is a basic concept called a “basic granule”, and then the builder method builds the family of basic concepts where an object may be included in many basic granules.

3.2.2. Granular Computing Network

Figure 5 depicts network techniques for granular computing. The training granular computing network algorithm is suggested by the following steps for training set: The atomic granule is first formed using the training set. Second, the stated union operator introduces the threshold $ϵ$ of granularity to conditionally combine the atomic granules, and the granule set is made up of all the union granules. Thirdly, the union process ends if all atomic granules are included in the GS granules; otherwise, the second phase is continued.

This stage computes the fitness value (i.e., Minimum Euclidean distance) of each basic concept to optimize the basic granules that are correlated to a specific class of decision attribute. The approach combines these basic granules to construct granule nodes in a granular network. The algorithm steps are shown in Figure 5.

The goal of the training process is to obtain the granule set and the related class label, which are used to predict the class label of an unknown input. The testing set, which is used to evaluate the effectiveness of granular computing network algorithms, is composed of testing data, including various data and their class labels. If the prediction class labels match the actual class labels, the testing data is classified. Otherwise, the testing data is incorrectly classified. One of the capabilities of granular classification algorithms is classification accuracy.

4. Experimental Results and Discussion

The purpose of this work is to evaluate the performance of the classification model for recognizing various kinds of human activities with the efficient placement of the sensor. The approach is compared in this experiment to traditional machine learning classifiers such as decision trees with a confidence factor of 0.25 and a minimum number of objects of 2, K-nearest neighbors (KNN) with a value of k = 49 [34], multi-layer perceptrons (MLP) with four hidden layers [35], support vector machines (SVM) with a sigmoid kernel function [5], naive bayes [36], and random forest with number of trees in the forest are 100, number of variables randomly sampled as candidates at each split are 100, maximum number of depth is unlimited, minimum number of objects per leaf node is 1, and minimum numeric class variance proportion is 0.001 [37].

4.1. Dataset Description

The Samsung Galaxy SII dataset from the UCI machine learning repository [2] was used to evaluate traditional machine learning classifiers as well as the approach. This dataset was collected from 30 volunteers aged 19–48 years. Each person wore a smartphone on their waists. Everyone had performed six activities. Moreover, the dataset consists of 10,299 observations and 561 attributes. The size of the dataset for each human activity in the training and testing datasets is described in

Table 1. Dataset size for each human activity.

Activity	Training Set	Testing Set
Laying	1407	537
Sitting	1286	491
Standing	1374	533
Walking	1226	496
Walking Downstairs	986	420
Walking upstairs	1073	471

.

4.2. Experimental Results on Approach

The training set contains 561 attributes. Preprocessing processes are applied to the training set to enhance its quality by removing noise. Moreover, the final number of attributes after performing feature selection is 105. The granular builder stage uses the optimal feature attributes to construct the basic granules and the basic concept family. Furthermore, the granular computing network employs the algorithm shown in Figure 5 to build the granular computing network with a threshold $ϵ$ of granularity from 0.2 to 0 with step 0.01. Furthermore, the approach discovered that the number of features influencing the decision attribute on the training set is six. Additionally, the classifier executes the optimal feature attributes of the testing set by using the testing algorithm shown in Figure 6 subset to measure the precision and recall of the approach.

4.3. Discussion

In this section, the approach for human activities using a granular computing classifier is evaluated versus traditional machine learning classifiers, i.e., KNN, Naive Bayes, MLP, Random Forest, and SVM. The granular computing classifier outperformed traditional machine learning classifiers. The evaluating metrics that are used in making comparisons include precision [38], recall [38], f-measure [38], and Matthews correlation coefficient (MCC) [39]. In fact, the approach can achieve correctly classified objects better than other traditional machine learning classifiers. As shown in

Table 2. Accuracy of Candidates Classifiers.

Classifier	Correctly Classified Objects	Incorrectly Classified Objects
KNN	2538	409
Naive Bayes	2475	472
MLP	2731	216
Random Forest	2674	273
SVM	2745	202
Peoposed Approach	2770	177

, the percentage of correctly classified objects using our approach is 94%. However, the percentages of correctly classified objects for KNN Naive Bayes, MLP, Random Forest, and SVM are 86.1%, 84%, 92.7%, 90.7%, and 93.2%, respectively.

Table 3. Detailed Precision by Class.

Classifier/Class	KNN	Random Forest	MLP	Naive Bayes	SVM	Approach
LAYING	1.000	1.000	0.994	0.993	1.000	1.000
SITTING	0.830	0.880	0.871	0.922	0.913	0.967
STANDING	0.752	0.853	0.905	0.711	0.865	0.960
WALKING	0.840	0.868	0.893	0.794	0.908	0.982
WALKING DOWNSTAIRS	0.930	0.972	0.985	0.845	0.973	0.977
WALKING UPSTAIRS	0.848	0.890	0.923	0.852	0.944	0.954
Overall Precision	0.865	0.910	0.928	0.853	0.933	0.973

shows the detailed accuracy of candidate classifiers per class. It consists of six activations: lying, running, standing, walking, stepping upstairs, and stepping downstairs. It reports the activity precision per row, and the last row concludes the overall precision of each algorithm over all activities. Table 3 emphasizes that the precision values of traditional machine learning classifiers are less than our approach in all activities as well as the overall precision shown in Figure 7.

Table 4. Detailed Recall by Class.

Classifier/Class	KNN	Random Forest	MLP	Naive Bayes	SVM	Approach
LAYING	1.000	1.000	0.998	0.998	1.000	1.000
SITTING	0.684	0.833	0.890	0.574	0.837	1.000
STANDING	0.870	0.895	0.878	0.930	0.927	1.000
WALKING	0.960	0.964	0.978	0.895	0.994	0.899
WALKING DOWNSTAIRS	0.786	0.840	0.917	0.752	0.929	0.817
WALKING UPSTAIRS	0.841	0.894	0.894	0.854	0.894	0.926
Overall Recall	0.861	0.907	0.927	0.840	0.931	0.940

,

Table 5. Detailed F-measure by Class.

Classifier/Class	KNN	Random Forest	MLP	Naive Bayes	SVM	Approach
LAYING	1.000	1.000	0.996	0.995	1.000	1.000
SITTING	0.750	0.856	0.880	0.708	0.874	0.983
STANDING	0.807	0.873	0.891	0.806	0.895	0.980
WALKING	0.896	0.913	0.934	0.842	0.949	0.939
WALKING DOWNSTAIRS	0.852	0.902	0.949	0.796	0.950	0.890
WALKING UPSTAIRS	0.844	0.892	0.908	0.853	0.918	0.940
Overall F-measure	0.860	0.907	0.927	0.836	0.931	0.955

and

Table 6. Detailed MCC by Class.

Classifier/Class	KNN	Random Forest	MLP	Naive Bayes	SVM	Approach
LAYING	1.000	1.000	0.995	0.994	1.000	1.000
SITTING	0.710	0.828	0.856	0.690	0.851	0.980
STANDING	0.763	0.845	0.868	0.767	0.871	0.976
WALKING	0.876	0.896	0.921	0.809	0.940	0.929
WALKING DOWNSTAIRS	0.833	0.890	0.942	0.766	0.942	0.878
WALKING UPSTAIRS	0.815	0.871	0.891	0.825	0.904	0.929
Overall MCC	0.835	0.890	0.912	0.812	0.918	0.948

compare among candidate activity recognition solutions in terms of recall, f-measure, and MCC based on each activity as well as overall recall, f-measure, and MCC values. These tables indicate that SVM outperforms the approach for two activities: walking and walking downstairs. However, the approach has the best recall, f-measure, and MCC for the other 4 activities as well as the best overall average of activities. Moreover, Figure 8, Figure 9 and Figure 10 depict bar chart comparisons of candidate solutions in terms of recall, f-measure, and MCC, respectively. Finally,

Table 7. Performance measure for each classifier.

Classifier/Class	Time in Seconds
KNN	5.55
Random Forest	5.83
MLP	440.97
Naive Bayes	0.97
SVM	1.86
Approach	4

indicates the overall processing time in seconds required for each classifier in both the learning and testing phases. The processing time of Naive Bayes is the minimum. However, the approach comes in third place after the Naive Bayes and SVM algorithms. Therefore, it has an acceptable processing time.

5. Conclusions

This article uses a granular computing classifier to recognize human activity. The approach consists of two phases: feature selection and classification. In the feature selection phase, the approach aims at identifying the most representative attributes that differentiate among different human activities. Feature selection tries to remove redundant and irrelevant attributes. On the other hand, the classification phase employs granular computing concepts to build a classification model to predict the expected label. Experimental results show that the approach significantly improves the classification accuracy of human activity recognition algorithms. Furthermore, the approach outperforms traditional classifiers in terms of precision, overall recall, overall f-measures, and overall MCC. Moreover, it has the highest recall, f-measures, and MCC for all activities except walking and walking down stairs. Although the processing time of the approach is not the best, it achieves a third lower processing time than SVM and Naive Bayes. Therefore, its processing time is acceptable with respect to other algorithms.