Next Article in Journal
Symmetries, Information and Monster Groups before and after the Big Bang
Previous Article in Journal
A Credit-Based Congestion-Aware Incentive Scheme for DTNs
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

PACP: A Position-Independent Activity Recognition Method Using Smartphone Sensors

1
School of Computer and Software, Nanjing University of Information Science & Technology, Nanjing 210044, Jiangsu, China
2
Jiangsu Collaborative Innovation Center on Atmospheric Environment and Equipment Technology, Nanjing 210044, Jiangsu, China
3
Jiangsu Engineering Center of Network Monitoring, Nanjing University of Information Science & Technology, Nanjing 210044, Jiangsu, China
*
Authors to whom correspondence should be addressed.
Information 2016, 7(4), 72; https://doi.org/10.3390/info7040072
Submission received: 21 October 2016 / Revised: 5 December 2016 / Accepted: 12 December 2016 / Published: 15 December 2016

Abstract

:
Human activity recognition has been a hot topic in recent years. With the advances in sensor technology, there has been a growing interest in using smartphones equipped with a set of built-in sensors to solve tasks of activity recognition. However, in most previous studies, smartphones were used with a fixed position—like trouser pockets—during recognition, which limits the user behavior. In the position-independent cases, the recognition accuracy is not very satisfactory. In this paper, we studied human activity recognition with smartphones in different positions and proposed a new position-independent method called PACP (Parameters Adjustment Corresponding to smartphone Position), which can markedly improve the performance of activity recognition. In PACP, features were extracted from the raw accelerometer and gyroscope data to recognize the position of the smartphone first; then the accelerometer data were adjusted corresponding to the position; finally, the activities were recognized with the SVM (Support Vector Machine) model trained by the adjusted data. To avoid the interference of smartphone orientations, the coordinate system of the accelerometer was transformed to get more useful information during this process. Experimental results show that PACP can achieve an accuracy over 91%, which is more effective than previous methods.

1. Introduction

Human activity recognition plays a significant role in many fields, especially in health care, disease control, sports and fitness. For example, activity recognition can be utilized for fall detection in the daily life of the elderly [1]. Once a fall is detected, an urgent message will be sent to the hospital or family members for help. It can be also used for estimating the calorie expenditure of activities in quotidian life and providing healthy recommendations for people [2]. Therefore, the ability to identify physical activity automatically and efficiently is necessary.
Existing works on activity recognition are divided into two categories according to the data source: vision-based and sensor-based. Vision-based activity recognition analyzes and recognizes the activity of humans automatically through the video or pictures captured by a fixed camera [3,4,5,6]. Vision-based activity recognition has good performance for an indoor environment; however, it is not widely used due to the high cost and environmental restriction. In addition, it may compromise the user’s privacy. The latter category is based on sensors like light sensor, accelerometer, gyroscope and magnetometer to get information from users [7,8,9,10]. Data-mining and machine-learning techniques such as clustering algorithms and classifiers are utilized for final recognition [11,12]. Compared to the former category, the sensor-based method is clearly better at adapting to environmental requirements. In the early stage of sensor-based research, wearable sensors were explored and proved to be feasible [13,14]. However, people would have to wear the sensors at all times, which is not convenient. With the popularity of the smartphone equipped with a variety of sensors, it is an attractive solution for activity recognition [15,16]. Some researchers identify activity with smartphones in an online manner, while most are offline [17]. The offline model can save energy consumption of smartphones’ batteries effectively, although there are still many challenges in this area, such as the orientation and position of a smartphone, the demographic of users, etc. [18]. To consider the position of a smartphone, most previous studies used a fixed position, such as trouser pockets, even though smartphone users are likely to hold it in their hands or elsewhere. This setting limits the freedom of the users. Reference [17] divided the position-independent activity recognition methods into four categories. It can be briefly divided into two solutions for this problem. One is to find some features that are insensitive to position, and the other is to identify the position and generate a tuple consisting of the position and activity. Neither of them considers using the information of position for more effective activity recognition, and their accuracy is thus only about 89% [19,20,21].
In this paper, we explore the impact of the smartphone position on activity recognition and try to use the information of position to improve the accuracy. In our work, we will examine four positions where people will usually place their smartphone: coat pocket, hand, trouser pocket and bag, and five daily activities: going upstairs, going downstairs, walking, running and standing. The first task is to recognize where the smartphone is with the accelerometer and gyroscope data. To avoid the influence of the orientation of the smartphone, the data from the accelerometer are processed before feature extraction. Then some adjustments on the accelerometer data are made corresponding to the position. Finally, we recognize activities with the adjusted data to improve the performance. The contributions of the paper are as follows:
  • The conversion of a smartphone coordinate system is analyzed and introduced to avoid the influence of smartphone orientation. Orientation is a big challenge during activity recognition. By coordinate system conversion in the Parameters Adjustment Corresponding to smartphone Position (PACP) method, tri-axial acceleration is transformed to vertical and horizontal acceleration. Vertical and horizontal acceleration are more useful because they can mostly reflect the motion of users.
  • Sensitive features are analyzed and selected for position recognition before activity recognition. All activities mentioned above and not only walking are studied [22]. Experiments show that these features have good performance.
  • According to the result of position recognition, some adjustments are made to improve the accuracy of activity recognition. Positon of smartphone can provide the information of the device. By adjusting some parameters, the differences between the same activity in different positions can be narrowed down, making activity recognition easier.
  • Both position recognition and activity recognition are investigated in our paper and all of them have high accuracies.
The remainder of this paper is organized as follows. Some related works are briefly introduced in Section 2. Section 3 explains the design of PACP in detail. Section 4 is the implementation and evaluation of PACP. We also discuss the findings and limitations of PACP in Section 5. Finally, in Section 6, it draws a conclusion of this paper.

2. Related Work

Many experts and researchers proposed their novel methods for activity recognition with smartphone sensors. Kwapisz et al. asked volunteers to carry a smartphone in their front pant leg pocket to get accelerometer data [23]. There were 43 features generated from the data to train a model for activity recognition and achieved accuracy above 90%. Some novel features were proposed like the time between peaks and binned distribution, but frequency domain features were not contained. The sensor data were also used for user authentication in the literature [24]. Bieber et al. fixed the smartphone in the trouser pocket and improved the capabilities of activity recognition by fusing the information of accelerometer and sound sensor (microphone), since the sounds can provide additional information about the user’s situation [25]. However, a microphone brings its own noise and there was no solution to this problem. Kwon et al. also used a specified position with a smartphone in a pant pocket [26]. Unsupervised learning techniques were used for activity recognition with smartphone sensors. In this way, it can solve the problem that researchers have of generating the labels. The method in [26] achieved 90% accuracy but had not researched going upstairs and downstairs, which are common everyday actions. Harasimowicz et al. used the accelerometer in the smartphone to classify eight user activities [27]: walking, sitting, going upstairs, going downstairs, standing, running, lying and even turning. They also investigated the influence of window size during recognition. In the best solution, KNN (k-Nearest Neighbor algorithm) resulted in an accuracy of 98.56%. The just obtained data only from the smartphone’s location in a trouser pocket, which limits the position. In addition, they did not describe the issue of smartphone orientation. Ustev et al. presented their algorithm of human activity recognition using the accelerometer, gyroscope and magnetic field sensors in smartphone [18]. They studied the challenges of users, device and device orientation. To avoid the impact of device orientation, they converted the coordinate system of the smartphone to earth’s magnetic coordinate system with the magnetic field vector. Experiments showed that their method works well. Shoaib et al. recognized complex human activity using both smartphone and wrist-worn motion sensors like a smartwatch [28]. Data from a smartphone in the right trouser pocket and a smartwatch on the right wrist were studied separately and combined to show the recognition performance. Their method could classify complex activities like smoking and talking but position and orientation of the smart device was also limited. All of these studies set the position of the smartphone in a fixed way for activity recognition and most of them obtained high accuracy. However, in reality, users usually do not fix their smartphone in one place.
When considering the orientation and position of the smartphone, the task becomes harder. Henpraserttae et al. conducted six daily activities with a smartphone in 16 different orientations and three different positions to study the influence of orientation and position [29]. They used a projection-based method to handle the various device orientations, but for positions, they used position-insensitive features to build models for each positon. Activity recognition in some positions had a low accuracy. Grokop et al. considered the variety of smartphone position [30]. They fused the data of the accelerometer, ALS (Ambient Light Sensor), proximity sensor and camera intensity to generate a tuple (motion state, device position) and, in this way, classified the motion states and device positions with the accuracies of 92.6% and 66.8%, respectively. Using more sensors might improve the accuracy while it could consume more resources of the smartphone. Moreover, this method also did not research going upstairs and downstairs which is similar to walking. Coskun et al. proposed a novel way to classify the position and activity by using solely an accelerometer. They recognized positions with angular acceleration to improve the position recognition accuracy but its activity recognition accuracy achieved only 85% [21]. Miao et al. placed smartphones in the two front and back pockets on the trousers and two front pockets on the coat for physical activity recognition [20]. They first used the light and proximity sensors to judge whether the smartphone was in the pocket or not; if so, then they collected raw data and extracted features which are insensitive to orientation and position for training models. Finally, they chose decision tree which has the best performance for classification and recognition. The resource of the smartphone can be saved by choosing the suitable situation to collect data and the recognition accuracy reached 89.6%. The method can deal with the smartphone orientation and position well but hand and bag were not considered in their study. In [19], Gu et al. explored the user-independent motion recognition using smartphone sensors. Different positions such as the trouser pocket, as well as holding with or without a swinging motion were studied during activity recognition. The barometer, which is newly embedded in the latest smartphones such as the iPhone 6, were used; the pressure derivatives captured by the barometer data can easily distinguish vertical activity from others. In order to improve the accuracy, the history results of activity recognition were utilized to make a decision and the final overall accuracy of different positions was 89.21%. Their method showed the good performance for activity recognition, but barometer is not a common sensor in most smartphones.
Fixed smartphone position during activity recognition will restrict the activity of users. As mentioned above, the performance of activity recognition needs to be improved by considering different positions. In this paper, the focus is on the influence of the position of the smartphone and use of the available information to improve activity recognition performance, which have not yet been taken into account.

3. The Design of PACP (Parameters Adjustment Corresponding to Smartphone Position)

The flowchart of the proposed activity recognition method, PACP, is shown in Figure 1. Similar to previous works [18,19,20,21,22], activity recognition can be divided into four steps: data collection, preprocessing, feature extraction and recognition. Firstly, raw data of smartphone sensors should be collected from users. In this paper, only accelerometer and gyroscope data are collected which are enough for recognition. The magnetic field sensor information has been discarded because it is strongly affected by the environment and is sometimes at risk of losing the data. Then PACP performs some operations to convert the smartphone coordinate system to a vertical and horizontal system with the accelerometer data in the preprocessing process. The raw data collected are formed into a data stream. The stream is separated with sliding windows to get some short time series data. After preprocessing, features are extracted from the segmentations. In contrast to previous studies, PACP first identifies the position of the smartphone, instead of recognizing the activity directly. For position recognition, position-sensitive features are selected, and then a decision tree—which proved to have a good performance in most previous works [19,20]—is trained for recognition. According to the result of position recognition, features extracted from acceleration are adjusted with a position parameter to improve the activity recognition accuracy. Finally, PACP constructs an SVM (Support Vector Machine) [31,32] model with adjusted features for activity recognition. PACP emphasizes the impact of smartphone position and characteristics of activities in different positions which is different from previous studies. As mentioned above, four positions (coat pocket, hand, trouser pocket and bag) and five activities (going upstairs, going downstairs, walking, running and standing) are studied in our work. In addition, six machine-learning algorithms are explored to find the best model for activity recognition.

3.1. Coordinate System Conversion

During data collection, raw tri-axial accelerometer and tri-axial gyroscope data are acquired from users’ smartphones for further activity recognition. The accelerometer measures velocity over time along three axes and the gyroscope measures the rate at which a device rotates around each of the three spatial axes. Acceleration data can reflect the state of motion and gyroscope data can reflect the state of the smartphone. However, as shown in Figure 2a, these data are relative to the coordinate system of the smartphone, and once the device orientation changes, the data of accelerometer and gyroscope will change too. That is to say, if users hold a smartphone in different orientations, the acceleration would change even if they are doing the same thing in the same way, thereby creating large problems for activity recognition. To avoid the impact of device orientation, the coordinate system is converted into vertical and horizontal directions to get more useful information in PACP. The acceleration of smartphone is transformed from tri-axial accelerations to vertical and horizontal accelerations. Instead of using magnetic field sensor data like [18], the task is done by using accelerometer only. Nowadays, developers do not need to filter the acceleration data to remove the gravity in three axes because they can get gravity and user acceleration separately [33]. As we all know, the gravity points toward the center of the Earth, so the user’s acceleration can be decomposed into two parts: one of them points toward the center of the Earth (like gravity) and the other is perpendicular to it. The angle between gravity and user’s acceleration can be calculated first, then user’s acceleration can be divided into vertical and horizontal acceleration easily using projection as follow:
cos θ = A g x A u x + A g y A u y + A g z A u z A g x 2 + A g y 2 + A g z 2 A u x 2 + A u y 2 + A u z 2
A v = A u x 2 + A u y 2 + A u z 2 cos θ
A h = A u x 2 + A u y 2 + A u z 2 sin θ
where θ is the angle between gravity and user’s acceleration. Agx, Agy and Agz represent the gravity in X, Y and Z axes. Aux, Auy and Auz represent the user’s acceleration in three axes, respectively. Av and Ah are the vertical and horizontal user accelerations. The conversion process is shown in Figure 2b.
By coordinate transformation, acceleration generated by users is converted to vertical and horizontal accelerations. These accelerations are only related to the users’ activities, which are not affected by smartphone orientation.

3.2. Position Recognition

The raw data from a smartphone are data streams. In order to extract features, these data are divided into various segmentations. A set of features such as mean, variance, correlation coefficient, zero crossing rate, maximum, etc. are calculated from these segmentations for recognition. To identify the positions efficiently, PACP chooses some features that are sensitive to position.

3.2.1. Selection of Sensitive Features

We use walking activity as a sample to explain these features. Figure 3 shows the walking waveforms of tri-axial gyroscope and user acceleration in vertical and horizontal orientations with a smartphone in four positions. It is obvious that the waveforms of the tri-axial gyroscope are different in the four positions. For example, when the smartphone is in a user’s hand, the changes of the gyroscope in three axes are gentle, whereas they are intense when placed in the user’s pockets. It is easy to understand why: When holding the smartphone in their hands, people either make it stationary relative to the body or swing it along with the arm while walking as per the user’s habits. The change rate of the gyroscope is smaller in hand than in other positions. When the smartphone is in the trouser pocket, the change rate is high.
In addition, there are also differences in users’ horizontal acceleration. According to Figure 3, it always has a higher value in the hand and the trouser pocket than the coat pocket and bag because of the movement of users’ hands and legs. The value changes smoothly when the smartphone is in the coat pocket. If the smartphone is in hand or trouser pocket, the dispersion degree of users’ horizontal acceleration is different. It can be used to distinguish these two positions.
Considering these factors, eight related features are chosen for position recognition:
  • The max value of vertical and horizontal acceleration
  • The max value of gyroscope (three axes)
  • The max value of change rate of the gyroscope of three axes
  • The standard deviation of horizontal acceleration
  • The mean of the angles between gravity and user acceleration
  • The range of horizontal acceleration.

3.2.2. Model Selection

According to the features described above, PACP constructs a decision tree to identify the positions of the smartphone. Decision tree is a classification and prediction model which represents the mapping mode between the feature vectors and object. It structures a binary tree by partitioning the attributes’ values and pruning according to the training set. It is easy to implement and understand the decision tree. In previous work such as [20], it also achieved good performance.

3.3. Activity Recognition

The goal of PACP is to identify activity effectively. The accuracy can be improved after the smartphone position is known. It chooses some features that are insensitive to position and adjusts some other features to reduce the influence of the position.

3.3.1. Feature Selection and Feature Adjustment

Figure 4 illustrates that the acceleration waveforms of the same activity have differences in different positions except for the vertical acceleration. The waveforms of vertical acceleration almost keep the same change trend for the same activity in different positions or we can say vertical acceleration is insensitive to the position. For different positions, vertical acceleration reflects the characteristics of motions. PACP does not choose gyroscope for activity recognition because the coordinate system and acceleration information are enough for the task.
As mentioned above, horizontal acceleration has obvious differences which are used for position recognition. If the differences of horizontal acceleration become smaller, the accuracy of activity might become higher. PACP therefore adjusts the horizontal acceleration before activity recognition. Because of the good performance of gyroscope data in reflecting the position information of smartphone, they are chosen to get a parameter for adjusting. The value should reflect the intensity of those activities in each position. The average value of the resultant gyroscope, the average value of the max value of gyroscope in three axes, as well as other values are calculated to test which one is best. Finally, the average value of the maximum change rate of the gyroscope in three axes is chosen as the position parameter for each position because it can reflect the position information best. PACP first computes the maximum change rate of the gyroscope in three axes of each sample and then calculates the average value of the rate of each position as a parameter. PACP describes it as α. These data α can reflect the intensity degree of movement in four positions which show that the trouser pocket has the highest score. The parameters are computed to adjust the horizontal acceleration in each position with Equation (4). This equation can narrow the differences of the horizontal acceleration between different positions. By doing so, features can reflect the characteristics of activities better. The adjusted value of horizontal acceleration is also used to calculate the resultant acceleration.
A h = ( 1 + 1 / α ) A h
Finally, nine features, which are extracted from adjusted accelerometer data and used for activity recognition, are described in Table 1.

3.3.2. Model Selection

Model selection is also an important factor of activity recognition. An appropriate model can save time and get a satisfactory accuracy. In this paper, we studied six kinds of machine-learning algorithms, which are Random Forest, Decision Tree, SVM (Support Vector Machine) [34], KNN (k-Nearest Neighbor) [16], Naive Bayes [20] and Discriminant Analysis. These algorithms are widely used in activity recognition and are always efficient. In PACP, we evaluate the recognition performance of each algorithm (details in Section 4). According to the experimental result, PACP chooses the SVM model, which has the best performance, for activity recognition.
SVM is a supervised learning model that is commonly used for pattern recognition, classification and regression analysis. It usually analyzes and solves linearly separable problems. For linearly non-separable cases, it uses a mapping algorithm to transform the linearly non-separable samples into high dimensional feature space and find a hyperplane for classification or regression. The objective function and associated constraints of SVM can be described as follows:
min ( 1 2 ω 2 + C i = 1 n ξ i )
s . t . , y i ( ω T x i + b ) 1 ξ i , i = 1 , . . . n
ξ i 0 , i = 1 , . . . n
The hyperplane is defined by ω Τ Χ + b = 0 . ω is the parameters matrix to define decision boundary between classes. T represents transpose of matrix. X is the samples. b is the intercept associated with decision boundary. C is a regularization parameter and ξ i is the slack variable to solve the problem of an outlier. yi is the class of ith sample. The Equations (6) and (7) describe constraints of Equation (5). Equation (5) consists of two terms. The first one is maximizing the margin between classes and the other is balancing the errors with C. The Equation (6) ensures the existence of a margin, and Equation (7) restricts the noise and specific points to make them linearly separable. When dealing with the linearly non-separable, it uses a kernel function to transform the samples into higher dimensional space. There are many available kernel functions including linear kernel function, quadratic kernel function, polynomial kernel function, radial basis function, multi-layer perceptron and so on.

4. Implementation and Evaluation

In this section, we present the implementation and evaluation of PACP. We first introduce the data collection platform of our experiment, then demonstrate the effectiveness of position recognition and activity recognition respectively in detail.
We evaluate the performance of PACP by Accuracy, Precision and Recall. They are defined as follows:
A c c u r a c y = T P + T N P + N × 100 %
P r e c i s i o n = T P T P + F P × 100 %
R e c a l l = T P T P + F N × 100 %
where TP means true positive or the number of correctly classified as a positive example. TN is true negative. TP + TN represents the number of samples that are correctly classified and P + N represents the total number of samples. Generally speaking, if the Accuracy is higher, the classifier will be better. In Equation (9), FP is false positive, which represents the number of incorrectly classified as a positive example. Precision reflects the proportion of classifying correctly in positive examples. For Recall, FN is false negative. Recall measures the recognition ability of the classifier to recognize the positive example.

4.1. Data Collection

We designed and developed an iOS app to collect raw accelerometer and gyroscope data from users, as shown in Figure 5. Ten volunteers, i.e., six males and four females, aged between 23 and 27 years old, were invited to place the smartphone in the coat pocket, hand, trouser pocket and bag, respectively. They carried out a series of activities including going upstairs, going downstairs, walking, running and standing. Each activity was required to last 2 min. During this process, the volunteers are allowed to place the smartphone in different orientations with their habits. The sampling frequencies of the accelerometer and gyroscope sensor data were both 32 Hz which would not bring the burden to CPU. To avoid the noise and the impact of the start and end of the activity, 0.5 minutes of data were cut off from the beginning and end of data stream, respectively; only 1 minute of data in the middle was left. For going upstairs or downstairs, more useless data were also cut off. These data were transmitted from the smartphone to the server through the network for further processing and analysis.
On the server which was a PC (Personal Computer), PACP preprocessed the raw data collected from volunteers. The sliding window technique was used to split the data stream into short segments. According to the research of [19], when the window size reaches 64, it obtained the best performance. PACP finally adopted 64 as the window size with 50% overlap to segment the sensor data. In this way, 2 s of data were dealt with for each time. Features were extracted from these segmentations for recognition.

4.2. Evaluation of Position Recognition

In Section 3, the method of position recognition was introduced in detail, including the transformation of the coordinate system as well as feature extraction. Features were extracted from the segmentations to generate sample feature vectors. During recognition, the four-fold cross-validation approach was used. The samples of all datasets are divided into four. Each time, one fold of these samples was treated as a test set and the rest were treated as a training set. The process is repeated four times and the average value is used for evaluation. A decision tree model was built for position recognition. Table 2 shows the confusion matrix of position recognition.
From the confusion matrix, we can find that features that PACP extracted are effective and PACP can achieve an accuracy of 93.76% in the position recognition process. It is easy to distinguish the position especially when the smartphone is held in the trouser pocket. There are also some samples that are not correctly identified, because there are also standing samples in our dataset. Standing samples are difficult to distinguish for the positions.
In [35], Incel presented a novel method for position recognition. Features extracted from accelerometer were investigated and proved to be enough for position recognition. Three datasets, which are collected from different positions during different activities, were studied and the method achieved a good performance. We extracted all motion-related, orientation-related and rotation-related features described in [35] and tested them in our dataset. The performance is shown in Table 3. It shows that PACP has a better performance.
The accuracy of PACP is a little higher than Incel’s method because PACP combined gyroscope with acceleration for position recognition.

4.3. Evaluation of Activity Recognition

4.3.1. Performance of PACP

As described in Section 3, PACP identified five activities after adjusting the features according to the results of position recognition. The standard of adjustment is as in Equation (4). For choosing the suitable parameter as α, PACP calculated four values for selection and finally chose the best one with decision tree. Table 4 shows the result of comparison between different values.
The average of maximum change rate of gyroscope was selected as α according to the result. In this paper, the experiment values of α are 2.8149 for the coat pocket, 1.0912 for hand, 3.4647 for the trouser pocket and 2.0351 for bag respectively. The trouser pocket has the highest value which means the movement in this position is the most active.
With the adjusted feature vectors, six kinds of machine-learning algorithms were used (including Random Forest, Decision Tree, SVM, KNN, Naive Bayes and discriminant analysis) to research the influence of models. For SVM, PACP used LIBSVM (A Library for Support Vector Machines) [36] for multi-classification problems and calculated the best parameters with the grid search method. The final parameters in LIBSVM are as follows: radial basis function for kernel function type, 32 for regularization parameter and 0.25 for gamma in kernel function. The accuracy results of these algorithms are shown in Figure 6.
It illustrates that Random Forest and SVM model showed better performance than other classifiers with accuracies of 91.06% and 91.27%. From the diagram, it is obvious that all the models obtain high accuracies. It shows the good performance of the features we chose. Finally, SVM—which has the highest recognition accuracy—was chosen for activity recognition in this paper. Table 5 shows the confusion matrix of activity recognition with SVM.
PACP can distinguish running and standing from other states easily because they have big differences in vertical acceleration. Since the acceleration characteristics are similar for going upstairs, going downstairs and walking, it is usually hard to identify the three states correctly. Our work can achieve a comparatively higher accuracy than previous work. If we just consider identifying these three activities, the accuracy is about 86% which is comparatively high.
In order to show that our method is position-independent, we also discuss the accuracy of activity recognition in different positions: the coat pocket, hand, trouser pocket and bag positions mentioned in our paper. Table 6 describes the results of activity recognition accuracy.
When the smartphone was placed in coat pocket and hand, the activity recognition accuracy was higher. The reason is that, in the coat pocket, the horizontal acceleration is small and when the smartphone is held in hand, people usually swing their arms with their body. Different people may have their own style which brings difficulty for recognition. After the adjustment of PACP, this problem is improved. Generally speaking, each position has good performance. PACP proposed in this paper can avoid the influence of smartphone positions effectively.

4.3.2. Comparison

To prove the effectiveness of PACP, some contrast experiments were carried out in this paper. An SVM model was built and tested utilizing four-fold cross-validation with unadjusted features just like PACP. The accuracy is used to show the improvement by adjusting the features. It is shown in Figure 7 that the accuracy was improved by about 1.3 points on average.
Miao’s method in [20] also studied activity recognition with smartphone sensors. The impact of both smartphone position and orientation were also researched in their paper. The activities they recognized including remaining static, walking, running, walking upstairs and walking downstairs similar to PACP. They presented some insensitive features to avoid the influence of smartphone position and orientation and finally built a decision tree for activity recognition. Considering the dataset and the good performance of their method, we implemented the algorithm they proposed in our dataset and compared the final performance with PACP in Table 7.
Comparing to Miao’s method, PACP has a better performance. Miao’s method focused on insensitive features to deal with the smartphone position and orientation, but PACP not only considered the insensitive features but also adjusted the difference between positions. The position information was used to improve the activity recognition performance, proving to be very effective.

5. Discussion

In PACP, the position information was used to improve the accuracy of activity recognition. Four positions and five activities were studied during this process. By coordinate system conversion in Section 3.1, the impact of orientations can be avoided. Change rate of gyroscope and horizontal acceleration were found to be sensitive to positions and used for classifying smartphone position. Adjustment according to the result of position proved to be very effective in PACP because the accuracy was improved by about 1.3 points. However, there are some limitations in PACP too. The adjustment is based on the result of position recognition. Because accuracy of position activity is not 100%, some data from different positions could be mistakenly classified into a particular position. In PACP, the position accuracy is about 94%. It is satisfied and can be improved in the future. The adjustment of horizontal acceleration is utilized to narrow the difference between the same activity in different positions, it can achieve this goal in PACP.

6. Conclusions

In this study, we presented a novel method called Parameters Adjustment Corresponding to smartphone Position (PACP) for position-independent activity recognition with smartphone sensors. Accelerometer and gyroscope, which are available for most smartphones, were chosen as sensors to study. Raw data were collected from users using an iOS app. Transforming raw acceleration data from smartphone coordinate system to vertical and horizontal accelerations can avoid the influence of orientations of smartphone. The change rate of gyroscope, horizontal accelerations and some other features were found sensitive to positions. The final accuracy of position recognition is up to 93.76% with four-fold cross-validation, which is relatively high. With the information of smartphone position, PACP adjusted horizontal acceleration data using position parameter. It narrowed the difference between the same motion in different positions. Experimental results show that the recognition accuracy of PACP is up to 91.27% using four-fold cross-validation, which is greatly improved by about 1.3 points compared to the unadjusted method. It proves that PACP is effective. We mainly considered the impact of smartphone position and studied five activities in this paper. In future work, more factors will be considered, such as the group of users. The elderly, for example, may have different movement patterns than young people. Power consumption is also an important factor. We will consider adjusting the sampling frequency dynamically to reduce energy consumption. In addition, we will try to study some more activities in the future.

Acknowledgments

This work is supported by the NSFC (National Science Foundation of China) (U1536206, 61232016, U1405254, 61373133, 61502242, 61173136), the Jiangsu Government Scholarship for Overseas Studies under Grant JS-2014-351, BK20150925, CICAEET (Jiangsu Collaborative Innovation Center on Atmospheric Environment and Equipment Technology) and PAPD (Priority Academic Program Development of Jiangsu Higher Education Institutions) fund.

Author Contributions

Rong Yang and Baowei Wang conceived and designed the experiments; Rong Yang performed the experiments; Rong Yang and Baowei Wang analyzed the data; Rong Yang wrote the paper. Both authors have read and approved the final manuscript.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. He, J.; Hu, C.; Li, Y. An Autonomous Fall Detection and Alerting System Based on Mobile and Ubiquitous Computing. In Proceedings of the 2013 IEEE 10th International Conference on Ubiquitous Intelligence & Computing and 2013 IEEE 10th International Conference on Autonomic & Trusted Computing, Vietri sul Mare, Italy, 18–21 December 2013; pp. 539–543.
  2. Saha, S. Human Activity Recognition and Estimation of Calorie Expenditure—A Data Analytic Approach. In Proceedings of the 2nd International Conference on Communication and Computing, Bengaluru, India, 3–5 December 2015.
  3. Lin, W.; Sun, M.T.; Poovandran, R.; Zhang, Z. Human Activity Recognition for Video Surveillance. In Proceedings of the International Symposium on Circuits and Systems, Seattle, WA, USA, 18 May–21 August 2008; pp. 2737–2740.
  4. Yin, J.; Meng, Y. Human Activity Recognition in Video using a Hierarchical Probabilistic Latent Model. In Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, San Francisco, CA, USA, 13–18 June 2010; pp. 15–20.
  5. Yu, C.; Wang, X.; Anwar, M.W.; Han, K. Multi-Features Encoding and Selecting Based on Genetic Algorithm for Human Action Recognition from Video. Res. J. Appl. Sci. Eng. Technol. 2013, 5, 5128–5132. [Google Scholar]
  6. Motwani, T.S.; Mooney, R.J. In Improving Video Activity Recognition using Object Recognition and Text Mining. In Proceedings of the 20th European Conference on Artificial Intelligence, Montpellier, France, 27–31 August 2012; pp. 600–605.
  7. Chernbumroong, S.; Atkins, A.S.; Yu, H. Activity Classification using a Single Wrist-worn Accelerometer. In Proceedings of the 2011 IEEE 5th International Conference on Software, Knowledge Information, Industrial Management and Applications, Benevento, Italy, 8–11 September 2011; pp. 1–6.
  8. Fujimoto, T.; Nakajima, H.; Tsuchiya, N.; Marukawa, H. Wearable Human Activity Recognition by Electrocardiograph and Accelerometer. In Proceedings of the 2013 IEEE 43rd International Symposium on Multiple-Valued Logic, Toyama, Japan, 22–24 May 2013; pp. 12–17.
  9. Zhan, Y.; Kuroda, T. Wearable Sensor-Based Human Activity Recognition from Environmental Background sounds. J. Ambient Intel. Humaniz. Comput. 2014, 5, 77–89. [Google Scholar] [CrossRef]
  10. Lara, O.D.; Labrador, M.A. A Survey on Human Activity Recognition using Wearable Sensors. IEEE Commun. Surv. Tut. 2013, 15, 1192–1209. [Google Scholar] [CrossRef]
  11. Gu, B.; Sun, X.; Sheng, V.S. Structural Minimax Probability Machine. IEEE Trans. Neural Netw. Learn. Syst. 2016, PP, 1–11. [Google Scholar] [CrossRef] [PubMed]
  12. Xia, Z.; Wang, X.; Sun, X.; Liu, Q.; Xiong, N. Steganalysis of LSB matching using differences between nonadjacent pixels. Multimed. Tools Appl. 2016, 75, 1947–1962. [Google Scholar] [CrossRef]
  13. Wang, Z.; Jiang, M.; Hu, Y.; Li, H. An Incremental Learning Method Based on Probabilistic Neural Networks and Adjustable Fuzzy Clustering for Human Activity Recognition by Using Wearable Sensors. IEEE Trans. Inform. Technol. B 2012, 16, 691–699. [Google Scholar] [CrossRef] [PubMed]
  14. Attal, F.; Mohammed, S.; Dedabrishvili, M.; Chamroukhi, F.; Oukhellou, L.; Amirat, Y. Physical Human Activity Recognition Using Wearable Sensors. Sensors 2015, 15, 31314–31338. [Google Scholar] [CrossRef] [PubMed]
  15. Wang, A.; Chen, G.; Yang, J.; Zhao, S.; Chang, C.Y. A Comparative Study on Human Activity Recognition Using Inertial Sensors in a Smartphone. IEEE Sens. J. 2016, 16, 4566–4578. [Google Scholar] [CrossRef]
  16. Jahangiri, A.; Rakha, H.A. Applying Machine Learning Techniques to Transportation Mode Recognition Using Mobile Phone Sensor Data. IEEE Trans. Intell. Transp. Syst. 2015, 16, 2406–2417. [Google Scholar] [CrossRef]
  17. Shoaib, M.; Bosch, S.; Incel, O.D.; Scholten, H.; Havinga, P.J. A Survey of Online Activity Recognition Using Mobile Phones. Sensors 2015, 15, 2059–2085. [Google Scholar] [CrossRef] [PubMed]
  18. Ustev, Y.E.; Incel, O.D.; Ersoy, C. User, Device and Orientation Independent Human Activity Recognition on Mobile Phones: Challenges and a Proposal. In Proceedings of the 2013 ACM Conference on Pervasive and Ubiquitous Computing Adjunct Publication, Zurich, Switzerland, 8–12 September 2013; pp. 1427–1436.
  19. Gu, F.; Kealy, A.; Khoshelham, K.; Shang, J. User-Independent Motion State Recognition Using Smartphone Sensors. Sensors 2015, 15, 30636–30652. [Google Scholar] [CrossRef] [PubMed]
  20. Miao, F.; He, Y.; Liu, J.; Li, Y.; Ayoola, I. Identifying typical physical activity on smartphone with varying positions and orientations. Biomed. Eng. Online 2015, 14, 1–15. [Google Scholar] [CrossRef] [PubMed]
  21. Coskun, D.; Incel, O.D.; Ozgovde, A. Phone Position/Placement Detection using Accelerometer: Impact on Activity Recognition. In Proceedings of the 2015 IEEE Tenth International Conference on Intelligent Sensors, Sensor Networks and Information Processing, Singapore, 7–9 April 2015; pp. 1–6.
  22. Fujinami, K. On-Body Smartphone Localization with an Accelerometer. Information 2016, 7, 21. [Google Scholar] [CrossRef]
  23. Kwapisz, J.R.; Weiss, G.M.; Moore, S.A. Activity Recognition using Cell Phone Accelerometers. ACM SIGKDD Explor. Newsl. 2010, 12, 74–82. [Google Scholar] [CrossRef]
  24. Kwapisz, J.R.; Weiss, G.M.; Moore, S.A. Cell Phone-Based Biometric Identification. In Proceedings of the 2010 Fourth IEEE International Conference on Biometrics: Theory Applications & Systems, Washington, DC, USA, 27–29 September 2010; pp. 1–7.
  25. Bieber, G.; Luthardt, A.; Peter, C.; Urban, B. The Hearing Trousers Pocket-Activity Recognition by Alternative Sensors. In Proceedings of the Petra 2011, the 4th International Conference on Pervasive Technologies Related to Assistive Environments, Crete, Greece, 25–27 May 2011; pp. 137–145.
  26. Kwon, Y.; Kang, K.; Bae, C. Unsupervised learning for human activity recognition using smartphone sensors. Expert Syst. Appl. 2014, 41, 6067–6074. [Google Scholar] [CrossRef]
  27. Harasimowic, A.; Dziubich, T.; Brzeski, A. Accelerometer-based Human Activity Recognition and the Impact of the Sample Size. In Proceedings of the 13th International Conference on Artificial Intelligence, Knowledge Engineering and Data Bases, Gdansk, Poland, 15–17 May 2014; pp. 130–135.
  28. Shoaib, M.; Bosch, S.; Incel, O.S.; Scholten, H.; Havinga, P.J. Complex Human Activity Recognition Using Smartphone and Wrist-Worn Motion Sensors. Sensors 2016, 16, 426. [Google Scholar] [CrossRef] [PubMed]
  29. Henpraserttae, A.; Thiemjarus, S.; Marukatat, S. Accurate Activity Recognition Using a Mobile Phone Regardless of Device Orientation and Location. In Proceedings of the 2011 International Conference on Body Sensor Networks, Dallas, TX, USA, 23–25 May 2011; pp. 41–46.
  30. Grokop, L.; Sarah, A.; Brunner, C.; Narayanan, V.; Nanda, S. Activity and Device Position Recognition in Mobile Devices. In Proceedings of the 13th International Conference on Ubiquitous Computing, Beijing, China, 17–22 September 2011; pp. 591–592.
  31. Gu, B.; Sheng, V.S. A Robust Regularization Path Algorithm for ν-Support Vector Classification. IEEE Trans. Neural Netw. Learn. Syst. 2016, PP, 1–8. [Google Scholar] [CrossRef] [PubMed]
  32. Gu, B.; Sheng, V.S.; Tay, K.Y.; Romano, W.; Li, S. Incremental Support Vector Learning for Ordinal Regression. IEEE Trans. Neural Netw. Learn. Syst. 2015, 26, 1403–1416. [Google Scholar] [CrossRef] [PubMed]
  33. Event Handling Guide for iOS. Available online: https://developer.apple.com/library/ios/documentation/EventHandling/Conceptual/EventHandlingiPhoneOS/motion_event_basics/motion_event_basics.html (accessed on 13 October 2015).
  34. Gu, B.; Sheng, V.S.; Wang, Z.; Ho, D.; Osman, S.; Li, S. Incremental learning for ν-Support Vector Regression. Neural Netw. 2015, 67, 140–150. [Google Scholar] [CrossRef] [PubMed]
  35. Incel, O.D. Analysis of Movement, Orientation and Rotation-Based Sensing for Phone Placement Recognition. Sensors 2015, 15, 25474–25506. [Google Scholar] [CrossRef] [PubMed]
  36. LIBSVM—A Library for Support Vector Machines. Available online: http://www.csie.ntu.edu.tw/~cjlin/libsvm/ (accessed on 12 January 2016).
Figure 1. Flowchart of the proposed Parameters Adjustment Corresponding to smartphone Position (PACP) activity recognition method.
Figure 1. Flowchart of the proposed Parameters Adjustment Corresponding to smartphone Position (PACP) activity recognition method.
Information 07 00072 g001
Figure 2. The coordinate system and conversion process: (a) The coordinate system of a smartphone; (b) The conversion process of acceleration.
Figure 2. The coordinate system and conversion process: (a) The coordinate system of a smartphone; (b) The conversion process of acceleration.
Information 07 00072 g002
Figure 3. The waveforms of acceleration and tri-axial gyroscope for walking in four positions: (a) Walking—in coat pocket; (b) in hand; (c) in trouser pocket; (d) in bag.
Figure 3. The waveforms of acceleration and tri-axial gyroscope for walking in four positions: (a) Walking—in coat pocket; (b) in hand; (c) in trouser pocket; (d) in bag.
Information 07 00072 g003
Figure 4. The waveforms of different activities in different positions: (a) Upstairs; (b) Downstairs; (c) Walking; (d) Running; (e) Standing.
Figure 4. The waveforms of different activities in different positions: (a) Upstairs; (b) Downstairs; (c) Walking; (d) Running; (e) Standing.
Information 07 00072 g004aInformation 07 00072 g004bInformation 07 00072 g004c
Figure 5. Data collection.
Figure 5. Data collection.
Information 07 00072 g005
Figure 6. Results of different algorithms.
Figure 6. Results of different algorithms.
Information 07 00072 g006
Figure 7. Comparison of unadjusted features and PACP.
Figure 7. Comparison of unadjusted features and PACP.
Information 07 00072 g007
Table 1. Features for activity recognition.
Table 1. Features for activity recognition.
NoFeaturesDescription
1disAccVRange of vertical acceleration
2maxAccVDifMax value of change rate of vertical acceleration
3accVTimeTime of vertical acceleration calculated with FFT
4engAccEnergy of resultant acceleration
5crossZeroRZero-crossing rate of vertical acceleration
6mAccMean value of resultant acceleration
7varAccVariance of resultant acceleration
8mAccHMean value of horizontal acceleration
9disAccRange of resultant acceleration
Table 2. Confusion matrix of position recognition.
Table 2. Confusion matrix of position recognition.
PredictedCoat PocketHandTrouser PocketBagRecall (%)
Actual
Coat pocket259242627493.57
Hand682548727792.15
Trouser pocket623226273995.18
Bag795825260394.14
Precision (%)92.5495.0794.2993.20
Table 3. Performance of two methods.
Table 3. Performance of two methods.
MethodsAccuracy (%)
Incel93.28
PACP93.76
Table 4. Result of comparison between different values.
Table 4. Result of comparison between different values.
αAccuracy (%)
The average of resultant gyroscope89.22
The average of maximum gyroscope in three axes 82.37
The average of change rate of gyroscope in three axes89.95
The average of maximum change rate of gyroscope in three axes90.59
Table 5. Confusion matrix of activity recognition.
Table 5. Confusion matrix of activity recognition.
PredictedGoing UpstairsGoing DownstairsWalkingRunningStandingRecall (%)
Actual
Going upstairs176319312822083.71
Going downstairs206174114316082.67
Walking10687207811091.06
Running141582245098.38
Standing00160226899.30
Precision (%)84.3985.5187.5797.86100
Table 6. Accuracy of different positions.
Table 6. Accuracy of different positions.
PositionAccuracy (%)
Coat pocket91.49
Hand93.01
Trouser pocket90.41
Bag89.94
Table 7. Performance of two methods.
Table 7. Performance of two methods.
MethodAccuracy (%)
Miao87.24
PACP91.27

Share and Cite

MDPI and ACS Style

Yang, R.; Wang, B. PACP: A Position-Independent Activity Recognition Method Using Smartphone Sensors. Information 2016, 7, 72. https://doi.org/10.3390/info7040072

AMA Style

Yang R, Wang B. PACP: A Position-Independent Activity Recognition Method Using Smartphone Sensors. Information. 2016; 7(4):72. https://doi.org/10.3390/info7040072

Chicago/Turabian Style

Yang, Rong, and Baowei Wang. 2016. "PACP: A Position-Independent Activity Recognition Method Using Smartphone Sensors" Information 7, no. 4: 72. https://doi.org/10.3390/info7040072

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop