Smartphone User Identification/Authentication Using Accelerometer and Gyroscope Data

Abstract

With the increasing popularity of smartphones, user identification has become a critical component to ensure security and privacy. This study looked into how smartphone sensors’ data can be utilized to identify/authenticate users and gives suggestions for the best application components. A public smartphone dataset was used to train a deep learning algorithms, conventional classifiers, and voting classifiers, which were then used to identify the users. Feature selection and Pre-processing techniques were investigated to improve the performance. According to the results, Recursive Feature Elimination beat the other feature-selection approaches, and Long Short-Term Memory (LSTM) had the best identification performance, as evidenced by a relatively large number of machine learning performance metrics. Even with a larger number of users, the proposed identification system performed well and outperformed existing approaches, which were primarily designed and tested on the same public smartphone dataset. In terms of user authentication, this study compared the effectiveness of accelerometer data against gyroscope data. According to the findings, the accelerometer data surpassed the gyroscope data in the authentication process. Notably, the study revealed that employing LSTM to combine the accelerometer and gyroscope data resulted in near-perfect user authentication. The insights gained from this study help to develop user identification and authentication approaches that employ smartphone accelerometer data.

1. Introduction

Smartphones have become an indispensable component of people’s lives, fulfilling a variety of functions other than communication. One notable application is in the Internet of Things (IoT), where smartphones use their data-sensing, visualization, and edge-computing capabilities to allow IoT-based solutions [1,2,3,4]. Likewise, smartphones serve as an entertainment platform, allowing users to engage in mobile games, as well as watch video and audio content [5,6]. They also function as dependable Global Positioning System (GPS) devices and navigation systems [7] and include services such as document scanning and imaging, allowing users to scan Quick Response (QR) codes and copy documents with ease [8]. Moreover, smartphones are equipped with built-in sensors that facilitate health and fitness monitoring, enabling users to track their physical activity and manage their health [9]. Another useful feature of smartphones is their capacity to support electronic payments, making mobile transactions and financial transactions more accessible [10]. Smartphones incorporate biometric authentication methods such as facial recognition [11], fingerprint identification [12,13], and palm-print identification [14,15], which are used not only for user recognition, but also for specialized applications such as identifying potential threats or terrorists [16,17].

Contemporary mobile phones are outfitted with a variety of sensors, which improve their operation and offer users a rich interactive experience. The accelerometer, gyroscope, and magnetometer are three important sensors found in modern mobile phones. The accelerometer detects changes in orientation and movement by measuring the phone’s linear acceleration in three dimensions. This sensor is in charge of counting steps, screen rotation, and recognizing gestures. The gyroscope supports the accelerometer by monitoring the rotational motion of the phone, allowing for more-exact tracking of movements and improving augmented reality apps. Finally, the magnetometer functions as a digital compass, sensing magnetic fields and assisting users in properly navigating their surroundings. Together, these sensors enable a wide range of applications and contribute to mobile phones’ general adaptability and utility [18,19,20,21].

With the growing popularity of smartphones, user identification has become an essential component of maintaining security and privacy. This research focused on the use of the accelerometer data obtained by a smartphone for user identification. The accelerometer sensor in smartphones records the device’s movements and orientation, which can be used to develop personalized profiles for various users.

Less research has been performed on the topic of identifying users using mobile phones’ accelerometer data. Although user authentication has seen much research using accelerometer data, as can be seen in the next section, presuming that each mobile phone has its own unique owner, this is not always the case, occasionally, several persons will use the same mobile device. Such scenarios include, but are not limited to use in education, sharing with friends and family, public use, business use, etc. For each of these scenarios, it is crucial to implement the necessary security precautions to safeguard each user’s privacy and personal information.

This paper attempted to investigate the potential applications of smartphone sensors’ data in depth, with an emphasis on user identification using accelerometer data. To begin, a variety of machine learning approaches, including deep learning, traditional classifiers, and voting classifiers, was used to learn from each user’s accelerometer data for precise identification. The key training resource for these methods was the Hand Movement, Orientation, and Grasp (HMOG) public dataset [22]. To improve the performance of machine learning, several Pre-processing approaches and feature-selection strategies were investigated to clean and optimize the data.

Building on the initial scope, the objectives of this study were expanded to encompass the process of authentication. The use of gyroscope data in conjunction with accelerometer data was investigated to determine their impact on authentication Accuracy. By leveraging both sensors, this study aimed to assess their combined effectiveness and potential synergies to improve user authentication.

This paper is organized into sections to give an in-depth investigation of smartphone user identification/authentication using sensor data. The “Related Work” Section 2 establishes the context for the study by providing an overview of previous literature and research pertinent to the scope of this paper.

The “Materials and Methods” Section 3 is broken into numerous subsections. The dataset utilized in the study is explained in Section 3.1, including information on the HMOG public dataset. Section 3.2 focuses on the strategies used to preprocess the accelerometer data to ensure their quality and suitability for analysis. In Section 3.3, the feature extraction procedure is described, along with the specific features extracted from the accelerometer signals. The feature-selection strategies used to determine the most-useful features are discussed in Section 3.4. The class imbalance issues are addressed in Section 3.5, along with techniques for dealing with this obstacle. Finally, in Section 3.6, the numerous machine learning classifiers utilized in the study are introduced, with specifics on how they were used in the user identification/authentication process.

The “Results and Discussion” section (Section 4) is divided into subsections to present the findings thoroughly. The comparison of filtered and unfiltered signals is examined in Section 4.1, along with their impact on the identification process. Section 4.2 provides the outcomes of the feature selection procedure. In Section 4.3, the classifiers are compared and analyzed to determine their Accuracy in identifying users. Section 4.4 focuses on authentication specifically, demonstrating the results obtained by merging the accelerometer and gyroscope data using LSTM. The findings of the study are compared to state-of-the-art approaches in Section 4.5, indicating the advances made in user identification and authentication. Finally, in Section 4.6, the study’s scientific contributions are summarized and stressed.

The paper concludes with the “Conclusions” Section 5, which summarizes the main findings and implications. This section emphasizes the study’s significance, provides insights into future research directions, and reaffirms the paper’s impact on the field of smartphone user identification and authentication.

2. Related Work

Modern smartphones frequently contain inertial sensors such as accelerometers, gyroscopes, and magnetometers, which has motivated researchers interested in authentication and identification techniques to take into account secure, dynamic behavioral biometrics based on these sensors. This section presents a number of works that dealt with behavioral biometric keys based on the data obtained from the aforementioned sensors.

An identification framework called ActID was presented by Sudhakar and his coworkers to identify a user based on hand movements made when the user is walking for 60 s on a flat surface. Activity sensors consisting of an accelerometer and gyroscope were placed on the wrist of the hand to collect data from 30 subjects over the course of two sessions. The sensors’ readings were subsequently sent through Bluetooth to a mobile phone [23]. The transmitted signals were then filtered and resampled using a linear interpolation method to replace missing data with the nearest value. Next, they extracted physical features such as the peak value, as well as statistical features such as the mean, median, and variance. Then, they used Optimal Feature Evaluation and Selection (OFES) Ram et al. [24] to choose features of high quality and Correlation-based Feature Subset Selection (CFSS) Hall [25] to choose features that were most-closely related to the labeled class. The results of various standard classifiers, including K-Nearest Neighbors (KNN), Naive Bayes (NB), and Random Forest (RF), were then applied to a subset of features, and their performance was compared with the sliding window voting classifier, which was based on SVM and had 100% Accuracy, outperforming the 97.98% Accuracy of the standard SVM classifier. However, the research was hindered by the fact that the available data did not encompass all user behaviors. Additionally, both the training and testing data were obtained from identical users, in identical circumstances, and over the same duration, which limited the study’s scope.

Cherifi and her coworkers [26] proposed an effective continuous authentication technique based on users’ prehensile movements and modeled these movements through the Hidden Markov Model-Universal Background Model (HMM-UBM) with continuous observations based on the Gaussian Mixture Model (GMM). Two datasets were used: the first was the HMOG public dataset, and the second was a proprietary database. The dataset was then filtered using the obfuscation method, which multiplied the original signal of each of the three employed sensors by an offset value and added new noise to the original signals. The sensor readings for each axis were then normalized by dividing their values by their magnitude and added as a fourth dimension. Multiple tests were conducted to determine the best window size, and a fixed sliding window was ultimately selected. The mean and standard deviation for each sensor axis and magnitude were the extracted features, resulting in a vector of twenty-four features. The proposed method achieved a 19.2% Equal Error Rate (EER) on the HMOG dataset and a 14.8% EER on their second dataset. This study neglected to account for the class-imbalanced HMOG dataset and lacked other performance metrics such as the Area Under the Curve (AUC), Specificity, etc. Sitova and coworkers [27] introduced a Biometric-Key-Generation (BKG) function on HMOG paired with taps and keystrokes, which was extracted by an accelerometer, gyroscope, and magnetometer from 100 users while they were seated and moving about. They extracted two types of features: The first type is known as grasp resistance, and it measures how quickly the phone moves and changes orientation in reaction to a tap gesture. The second type, known as grasp stability, measures how fast tap forces cause changes in movement and orientation. The extracted features for both types included sensor readings on the x, y, and z, magnitude, the start and end times of the taps, the stability after the tap ends, and the average sensor reading in 100ms. Using combined accelerometer and gyroscope data, they were able to obtain a 13.62% EER on HMOG features. However, the EER dropped to 7.16% when they integrated HMOG, taps, and keystrokes. Since HMOG is a class-imbalanced dataset, other metrics, such as the Precision and Recall of the minority class, must be used instead since their model was only evaluated using one metric (EER).

A study by Yoneda and coworkers [28] provided a thorough investigation of mobile biometrics by evaluating nine sensors and 18 physical activities. The data were collected at 20 Hz by the accelerometer and gyroscope incorporated into inertial smartwatches and smartphones. About three minutes were spent performing each of the 18 activities by the 51 users who made up the study’s sample. Every activity was split into eighteen instances with a defined window size of ten seconds, which did not overlap. For each instance for both sensors, the statistical features of the Average, Standard Deviation, Average Absolute Difference, Time between Peaks, Average Resultant Acceleration, Binned Distribution, Activity label, and Subject ID were retrieved. The KNN, Decision Tree, and Random Forest were investigated. The accelerometer sensor combinations for both the watches and the phones, together with all other sensor combinations utilizing Random Forest with voting yielded the lowest EER for an authentication of 9.3%. However, the authentication Accuracy increased to 99.7% by combining Random Forest with voting based only on the accelerometer of both the smartphones and smartwatches used.

An authentication approach based on a fingertip sensor device that collects motion data such as angular velocity and acceleration, as well as physiological data such as a Photoplethysmography (PPG) signal was presented by Wu and coworkers [29]. The compact and portable fingertip device had two sensor chips: the Flora chip detects movement, and the Pulse Sensor Amped chip checks blood oxygen levels. As soon as it is worn, it starts capturing acceleration, angular velocity, and PPG signals. These data are then transmitted through Bluetooth to a computer. The data were collected from 40 subjects, who agreed to wear the fingertip device while performing three actions—slow walking, sitting, and performing rather hard movements—for 20 repeats (each repetition lasting 12 s) over the course of 30 days. They used a fourth-order Chebyshev low-pass filter with a cutoff frequency of 5 Hz to decrease the noise in the data. Then, a peak detection method was used to extract the periodicity of PPG signals, and a Fixed-size Sliding Window (FSW) ranging from 2 s to 12 s with 20% overlap was applied to extract the statistical features, which were divided into three categories: time-domain, frequency-domain, and wavelet-domain features. Then, they used the ReliefF algorithm [30] to determine the importance of the features, choosing 90% of them that were more crucial for authentication. For activity recognition, the Decision Tree performed the lowest with an Overall Accuracy of 94.3%, while the KNN earned the best performance with 100% Accuracy. With a False Acceptance Rate (FAR) of 4.69%, a False Rejection Rate (FRR) of 4.95%, and 98.74% Accuracy, the walking state learned by the Support Vector Machine (SVM) provided the best authentication performance.

The works of [31,32,33,34,35,36,37,38,39,40,41,42,43,44] are other examples of research that investigated the use of smartphone accelerometer data for biometrics.

The bulk of previous research, as can be seen, only used one evaluation metric, leaving out crucial metrics such as the Area Under the Curve (AUC), Index of Balanced Accuracy (IBA) [45], Sensitivity, Specificity, Geometric Mean, and F-Score. Furthermore, the majority of these studies relied on person authentication rather than identification, and when they addressed the issue of class imbalance, they employed the oversampling approach, which creates new examples based on their resemblance to the minority examples in question and assumes that they belong to the same class, even though this is not always the case according to [46].

Table 1. Summary of some related literature: number of users, feature-selection methods, classifiers, and performance metrics. Optimal Feature Evaluation and Selection (OFES), Correlation-based Feature Subset Selection (CFSS), Scaled Manhattan (SM), Scaled Euclidean (SE), Gaussian Naive Bayes (GNB), Stochastic Gradient Descent (SGD), Accuracy (Acc), Decision Tree (DT), Stochastic Gradient Descent (SGD), Voting Classifier (VC).

Work	Users	Sensors	Feature Selection	Classifiers	Metrics
[23]	30	Accelerometer Gyroscope	OFSS CFSS	KNN NB RF SVM	Acc
[26]		Accelerometer Gyroscope Gravity	-	HMM-UBM	EER
[27]	100	Accelerometer Gyroscope Magnetometer	-	SVM SM SE	EER
[29]	40	Flora Chip Pulse Sensor Amped Chip	ReliefF	KNN DT SVM	FAR FRR
[28]	51	Accelerometer Gyroscope	-	KNN DT RF with Voting	EER Acc
[47]		Accelerometer Gyroscope Magnetometer Touch Screen	-	Network with Three Dense Layers	EER Acc Pre
Proposed	10 20 30 40 50	Accelerometer Gyroscope	CC FS IG RFE All Features	KNN RF DT GNB SGD ANN VC XGBoost LSTM	Acc Pre Rec F1 GEO IBA AUC

lists some of the studies with information about their experiments’ setup.

This paper focused on these drawbacks and worked on person identification using a variety of widely used evaluation metrics, including the AUC, IBA, GEO, and F-Score. Additionally, it looked into the use of Extreme Gradient Boosting (XGBoost) [48], which has been recommended for the class imbalance problem [49], as well as other approaches, such as [50].

3. Materials and Methods

Data preparation, feature extraction, feature selection, and training/testing using a machine learning classifier were the four main components of the proposed identification system, as illustrated in Figure 1.

The proposed system leverages data obtained from the mobile accelerometer sensor, which is made up of a set of slender structures and is used for motion detection. These structures produce readings, which are then transmitted to the main circuit. Tri-axial accelerometers are sensors that approximate acceleration in the x, y, and z axes numerically. This information can be used to calculate velocity and displacement [51].

3.1. Dataset Description

Hand Movement, Orientation, and Grasp (HMOG) [22,27] is a dataset of features captured from human interaction with smartphones such as holds, taps, and grasps. Accelerometer, gyroscope, and magnetometer sensors were used to capture the subtle hand movements and orientation while users interacted with their smartphones. The HMOG dataset was captured from 100 subjects (53 male and 47 female) over the course of eight sessions, with four sessions involving the participants writing a text of at least 250 characters while they were walking and the remaining four with the volunteers writing a text while sitting.

The data were collected using Android Samsung Galaxy S4 for all volunteers to record two types of readings:

• Accelerometer, gyroscope, and magnetometer readings with a sampling rate of 100 Hz.
• Touch screen data such as touch gestures and key press.

This study utilized the data obtained from the accelerometer sensor only to identify users. Figure 1 shows a sample of the accelerometer sensor reading in one second, and

Table 2. Summary of HMOG dataset characteristics.

Category	Content
Accelerometer	Timestamp, acceleration force along x/y/z axis
Subjects	53 (males), 47 (females)
Number of Examples/Subject (Class)	in the range of 1162 to 4639

summarizes the dataset characteristics.

3.2. Preprocessing

The goal of this step was to clean and prepare the sensor’s raw data so that the feature extraction process could make efficient use of them. The preprocessing steps included calculating the magnitude of the three axes (x, y, and z) for each signal, bandpass filtering the data, and utilizing an overlapping sliding window to extract the signal’s internal features. Equation (1) can be used to more easily and effectively calculate the magnitude of the x, y, and z vectors of the accelerometer sensor than it is to calculate each axis separately.

$$M=\sqrt{x^2+y^2+z^2},$$

(1)

where M is the magnitude and x, y, and z are the acceleration at a specific timestamp on the three axes of the accelerometer signal.

It is usually recommended to use a bandpass filter to reduce noise in a signal [52], although this largely depends on the amount of noise that may be present in the HMOG dataset. This study tested two different scenarios, the first of which made use of a bandpass filter that was applied at low and high frequencies between 3 and 8 Hz, which corresponds to the acceptable range for human hand tremors [53]. The second scenario was to omit this step in the event that the data were noise-free.

Utilizing the overlapping sliding window technique for preprocessing low-dimensional time series data results in reduced latency, improved Accuracy, and lower processing costs, particularly when employing a smaller window size [54]. In order to improve the Accuracy, reduce the feature extraction time, and give more comprehensive user data so the model can learn from them, each user signal was split into small segments in this study using the overlapping sliding window technique. The window size was fixed at 10 s (10 s × 100 samples per second = 1000 samples), as illustrated in Figure 2.

3.3. Feature Extraction

One of the most-important processes in building a trained model is feature extraction since choosing the best features enables the model to train more smoothly and accurately differentiate between numerous classes. The Fast Fourier Transform (FFT) was used to transform the row magnitude signal from the time domain to the frequency domain for each sliding window. The statistical features, which were extracted from the resultant FFT signal and the pure magnitude signal, are listed in

Table 3. The extracted statistical Features from both signals: the Magnitude (Mag), and the Fast Fourier Transform (FFT).

Feature	Description	Equation
Above mean (Mag)	Count of acceleration measurements that are larger than the mean acceleration value of a corresponding window.	$$\begin{array}{c}\hfill |W^{+}|:W^{+}=\left\{\begin{array}{c}{\displaystyle a_tϵW:a_t>\left(\frac{1}{W_l}\sum _{t=0}^{W_l}{a}_t\right)}\end{array}\right\}\end{array}$$
Above mean (FFT)	Count of frequencies that are larger than the mean frequency value of a corresponding window.	$$\begin{array}{c}\hfill |W^{+}|:W^{+}=\left\{\begin{array}{c}{\displaystyle f_tϵW:f_t>\left(\frac{1}{W_l}\sum _{t=0}^{W_l}{f}_t\right)}\end{array}\right\}\end{array}$$
Below mean (Mag)	Count of acceleration measurements that are smaller than the mean acceleration value of a corresponding window.	$$\begin{array}{c}\hfill |W^{-}|:W^{-}=\left\{\begin{array}{c}{\displaystyle a_tϵW:a_t<\left(\frac{1}{W_l}\sum _{t=0}^{W_l}{a}_t\right)}\end{array}\right\}\end{array}$$
Below mean (FFT)	Count of frequencies that are smaller than the mean frequency value of a corresponding window.	$$\begin{array}{c}\hfill |W^{-}|:W^{-}=\left\{\begin{array}{c}{\displaystyle f_tϵW:f_t<\left(\frac{1}{W_l}\sum _{t=0}^{W_l}{f}_t\right)}\end{array}\right\}\end{array}$$
Complexity-invariant distance (Mag)	Calculate how complex the time series is (number of peaks, valleys, etc.).	$$\begin{array}{c}\hfill CID=\sqrt{{\sum }_{t=1}^{w_l-1}{(a_t-a_{t+1})}^2}\end{array}$$
Complexity-invariant distance (FFT)	Calculate how complex the frequency series is (number of peaks, valleys, etc.).	$$\begin{array}{c}\hfill CID=\sqrt{{\sum }_{t=1}^{w_l-1}{(f_t-f_{t+1})}^2}\end{array}$$
Skewness (Mag)	Measure of the symmetry at the right and the left of the center point of the window.	$$\begin{array}{c}\hfill {\gamma }_l=\frac{{\sum }_{t=0}^{W_l}(a_t-{\overline{a}}_w)}{W_l{\left(s_w\right)}^3}\end{array}$$
Skewness (FFT)	Measure of the symmetry at the right and the left of the center point of the window.	$$\begin{array}{c}\hfill {\gamma }_l=\frac{{\sum }_{t=0}^{W_l}(f_t-{\overline{f}}_w)}{W_l{\left(s_w\right)}^3}\end{array}$$
Standard deviation (Mag)	The average amount of how far each acceleration value lies from the mean acceleration.	$$\begin{array}{c}\hfill s_w=\sqrt{\frac{{\sum }_{t=0}^{W_l}{(a_t-{\overline{a}}_w)}^2}{W_l-1}}\end{array}$$
Standard deviation (FFT)	The average amount of how far each frequency value lies from the mean frequency.	$$\begin{array}{c}\hfill s_w=\sqrt{\frac{{\sum }_{t=0}^{W_l}{(f_t-{\overline{f}}_w)}^2}{W_l-1}}\end{array}$$
Sample entropy (Mag and FFT)	The complexity of physiological frequency series signals.	$$\begin{array}{c}\hfill SampleEn(m,r)={log}_e\left(\frac{A^{m+1}\left(r\right)}{A^m\left(r\right)}\right)\end{array}$$
Maximum acceleration (Mag)	The maximum magnitude value.	$$max\left(a\right)=\underset{t=0}{\overset{W_l}{\mathrm{m}\mathrm{a}\mathrm{x}}} a_t$$
Maximum frequency (FFT)	The maximum frequency value.	$$max\left(f\right)=\underset{t=0}{\overset{W_l}{\mathrm{m}\mathrm{a}\mathrm{x}}} f_t$$
Median (Mag)	The middle number in the sorted magnitude values.	$\{\begin{array}{cc}{a}_t^{\left(i\right)}\hfill & :i=\frac{W_l^{\left(\mathcal{O}\right)}+1}{2}\hfill \\ \frac{a_t^{\left(i\right)}+a_t^{(i+1)}}{2}\hfill & :i=\frac{W_l^{\left(\mathcal{E}\right)}}{2}\hfill \end{array}$
Median (FFT)	The middle number in the sorted frequency values.	$\{\begin{array}{cc}{f}_t^{\left(i\right)}\hfill & :i=\frac{W_l^{\left(\mathcal{O}\right)}+1}{2}\hfill \\ \frac{f_t^{\left(i\right)}+f_t^{(i+1)}}{2}\hfill & :i=\frac{W_l^{\left(\mathcal{E}\right)}}{2}\hfill \end{array}$
Mean (Mag)	Average of the magnitude in a corresponding window.	${\displaystyle \frac{1}{W_l}\sum _{t=0}^{W_l}{a}_t}$
Mean (FFT)	Average of the frequencies in a corresponding window.	${\displaystyle \frac{1}{W_l}\sum _{t=0}^{W_l}{f}_t}$

and

Table 4. The extracted statistical features from both signals: the magnitude (Mag) and the Fast Fourier Transform (FFT)—continued.

Feature	Description	Equation
Kurtosis (Mag)	Measure of acceleration outliers in a corresponding window.	${\displaystyle {\beta }_2=\frac{1}{W_l}\frac{{\displaystyle \sum _{t=0}^{W_l}}{\left(a_t-{\overline{a}}_w\right)}^4}{s_w^4}}$
Kurtosis (FFT)	Measure of frequency outliers in a corresponding window.	${\displaystyle {\beta }_2=\frac{1}{W_l}\frac{{\displaystyle \sum _{t=0}^{W_l}}{\left(f_t-{\overline{f}}_w\right)}^4}{s_w^4}}$
Autocorrelation (Mag)	Serial correlation value with lag = 1 of accelerations within a corresponding window.	${\displaystyle \widehat{R}\left(l\right)=\frac{1}{\left(W_l-l\right)s_w^2}\sum _{t=0}^{W_l-l}\left(a_t-\overline{a}\right)\left(a_{t+l}-{\overline{a}}_w\right)}$
Autocorrelation (FFT)	Serial correlation value with lag = 1 of frequencies within a corresponding window.	${\displaystyle \widehat{R}\left(l\right)=\frac{1}{\left(W_l-l\right)s_w^2}\sum _{t=0}^{W_l-l}\left(f_t-\overline{f}\right)\left(f_{t+l}-{\overline{f}}_w\right)}$
Sum of absolute differences (Mag)	Sum over the absolute value of consecutive changes in the magnitude series.	${\displaystyle SAD=\sum _{t=0}^{W_l-1}\left|a_{(t+1)}-a_t\right|}$
Sum of absolute differences (FFT)	Sum over the absolute value of consecutive changes in the frequency series.	${\displaystyle SAD=\sum _{t=0}^{W_l-1}\left|f_{(t+1)}-f_t\right|}$
Energy (Mag)	Sum of all squared accelerations for the corresponding window.	${\displaystyle E=\sum _{t=0}^{W_l}{a}_t^2}$
Energy (FFT)	Sum of all squared frequencies for the corresponding window.	${\displaystyle E=\sum _{t=0}^{W_l}{f}_t^2}$
Peaks (Mag)	Number of peaks (values bigger than its 2 neighbors).	${\displaystyle E=\sum _{t=0}^{W_l}{f}_t^2}$
Amplitude of peak power spectral density (Mag)	Amplitude of the maximum signal power.	${\displaystyle A_{\left(PS{D}_P\right)}=\underset{a_t\in W}{max}\left(\sqrt{\frac{1}{W_l}{\left|\sum _{t=0}^{W_l-1}{a}_t{e}^{\left(\frac{-j2\pi kt}{W_l}\right)}\right|}^2}\right)}$
Median frequency (FFT)	Frequency dividing the total power area into two equal amplitude parts.	${\displaystyle f_{med}:\left(\sum _{f=f_l}^{f_{med}}PSD\right)=\left(\frac{1}{2}\sum _{f=f_l}^{f_h}PSD\right)}$
Frequency dispersion (FFT)	Dispersion of sample frequencies within a corresponding window.	${\displaystyle f_{disp}=2f_{step}:\left(\sum _{f_{med}-f_{step}}^{f_{med}+f_{step}}PSD=\frac{68}{100}\sum _{f=f_l}^{f_h}PSD\right)}$
Fundamental frequency (FFT)	Frequency that carries the maximum energy.	${\displaystyle f_{fund}:PS{D}_{fund}=\underset{f_l}{\stackrel{f_h}{max}}\left\{PSD\right\}}$
Frequency difference (FFT)	Sum over the difference value of consecutive changes in the frequency series.	${\displaystyle f_{\Delta }=f_{med}-f_{fund}}$
Spectral centroid amplitude (FFT)	Weighted average of the amplitude spectrum of frequencies of a corresponding window.	${\displaystyle SCA=\frac{{\displaystyle \sum _{f=f_l}^{f_h}}\left(f\right)\left(PSD\right)}{{\sum }_{f=f_l}^{f_h}\left(f\right)}}$
Maximum weighted PSD (FFT)	The maximum spectral energy of the frequency component within the corresponding window.	${\displaystyle PSD{\left(w\right)}_{max}=\underset{f_l}{\stackrel{f_h}{max}}\left\{\left(f\right)\left(PSD\right)\right\}}$

.

3.4. Feature Selection

A subset of features that were relevant from a vast number of features was chosen via a feature-selection technique. The primary goal of this step was to improve the machine learning model’s assessment metrics by deleting unnecessary or redundant data. The experiments were conducted with a number of feature-selection methods to assess and determine the most-prevailing features. These methods included the Correlation Coefficient, Fisher Score, Information Gain, and Recursive Feature Elimination (RFE). The chosen subsets of features from different techniques were tested using different classifiers. The outcomes of these experiments will be discussed in the next section.

The features that had a Correlation Coefficient greater than 0, as shown in Figure 3, were chosen for testing with various classifiers for identifying users. The outcomes of these tests were then compared to the results obtained from other feature-selection methods and with the results obtained from all features under identical testing and classifier settings.

The Fisher Score can also be utilized to determine the importance of features. A different subset of features of top importance (10, 20, and 30 features) was chosen and subjected to the same classifiers for identifying users. The results were compared to those produced by classifiers that utilized all features to determine which approach yielded better results. Figure 4 shows the features with the Fisher Score ranks.

The Information Gain was also investigated as a feature-selection method in this study. Specifically, features with an Information Gain value greater than 0.1 and 0.15 were selected. The Information Gain of each feature is shown in Figure 5.

The ranking of features based on their importance is shown in Figure 6, as determined by the RFE, which eliminates the least-important features. Logistic Regression (LR) and Random Forest (RF) were the two models used to determine the importance of the features. The effectiveness of this method was assessed by multiple classifiers, both with 10 and 20 features removed.

3.5. Class Imbalance

As can be seen from Table 2, HMOG is a class-imbalanced dataset. Usually, researchers opt for oversampling techniques that synthesize new examples for the minority classes/subjects (those having fewer accelerometer signals) based on their similarity to the other examples belonging to the same class/subject. However, this assumption might be violated according to a recent study [46], and it is recommended to avoid such an approach, stressing the use of other resampling approaches such as the ensemble approach [50,55].

This work investigated the use of XGBoost [48] because it provides the ability to adjust the training process to focus more on the misclassification of the minority class in class-imbalanced datasets [49,56].

3.6. Machine Learning Classifiers

In order to determine the best option for the proposed identification system, a variety of traditional classifiers, voting classifiers, and deep learning algorithms were investigated. The traditional classifiers include Artificial Neural Networks (ANNs), Decision Trees (DT), Random Forests (RFs), SVMs, KNNs, Logistic Regression (LR), Stochastic Gradient Descent (SGD), and Gaussian Naive Bases (GNB).

The Voting Classifier (VC) is an ensemble learning method that selects the class with the most votes among various classifiers as the prediction of the ensemble model. In this study, the voting classifier selected the most-voted class among the top three traditional classifiers, which are the ANN, SVM, and RF.

The second voting classifier was the well-known XGBoost, a machine learning method that builds machine learning models using the gradient boosting framework, which is a common framework. The gradient boosting framework combines a number of Decision Trees, which are weak learners, to produce a strong learner that can predict outcomes accurately. XGBoost entails successive iterations of adding new models to the ensemble and adjusting the weights of the data points based on the mistakes made by the prior models; this is vital for the misclassifications of the minority examples.

The Long Short-Term Memory (LSTM) was used as a deep learning classifier, which is a type of Recurrent Neural Network (RNN), which can process single and entire sequences of data.

4. Results and Discussion

A set of experiments was conducted to look into various concerns in order to evaluate the proposed user-identification system, including the following experiments:

• Comparisons of identification results on filtered and unfiltered signals to recommend the best alternative for the proposed system.
• Feature-selection comparisons to identify what most-effective attributes that can boost the identification metrics.
• Comparing classifier performance to determine the most-suitable classifier and alternatives for the proposed system.

4.1. Filtered vs. Unfiltered Signals

In this set of experiments, 10 randomly chosen users were identified using their accelerometer signals using 8 conventional classifiers and 2 voting classifiers, with and without bandpass filtering. Figure 7 shows the classifiers’ highest levels of Accuracy. This figure illustrates how significantly filtering the HMOG data reduced the performance of the identification process, giving the impression that the data were not noisy and did not require filtering because doing so would result in the loss of some crucial information from the original accelerometer signal, as the desired information would be sacrificed in order to reduce the noise. This is also supported by many studies such as [57] and Figure 8, which depicts the AUC of identifying 10 users using RF classifier.

As demonstrated in

Table 5. All classifiers’ performance on filtered and unfiltered signals.

Classifier	Without Filter						With Filter
	Pre	Rec	Spe	F1	GEO	IBA	Pre	Rec	Spe	F1	GEO	IBA
ANN	0.85	0.85	0.98	0.85	0.91	0.82	0.61	0.61	0.95	0.61	0.76	0.56
DT	0.75	0.75	0.97	0.75	0.85	0.71	0.45	0.44	0.94	0.44	0.64	0.39
GNB	0.5	0.5	0.94	0.46	0.65	0.44	0.31	0.3	0.91	0.29	0.48	0.25
KNN	0.83	0.83	0.98	0.83	0.9	0.8	0.5	0.49	0.94	0.48	0.67	0.44
LR	0.63	0.63	0.95	0.62	0.77	0.58	0.49	0.48	0.94	0.48	0.67	0.43
RF	0.84	0.84	0.98	0.84	0.91	0.81	0.59	0.59	0.95	0.59	0.74	0.54
SGD	0.55	0.54	0.94	0.53	0.71	0.49	0.43	0.42	0.93	0.41	0.62	0.37
SVM	0.85	0.85	0.98	0.85	0.91	0.82	0.55	0.55	0.95	0.54	0.72	0.5
Voting	0.87	0.87	0.98	0.87	0.92	0.84	0.61	0.61	0.95	0.61	0.76	0.56
XGBoost	0.88	0.88	0.99	0.88	0.93	0.86	0.6	0.6	0.95	0.6	0.75	0.55

, unfiltered data were better for identification, not only in terms of identification Accuracy, but also in terms of other crucial metrics such as the Precision (Pre), Recall (Rec), Specificity (Spe), F-Score (F1), Geometric Mean (GEO), and Index of Balanced Accuracy (IBA).

4.2. Feature-Selection Results

ANN, RF, SVM, VC, and XGBoost were used to evaluate the impact of certain standard feature-selection methods, including the Correlation Coefficients, Information Gain, Fisher Score, and RFW, as they achieved the most-favorable outcomes in the previous set of assessments. The identification Accuracy of several scenarios, where various subsets of features were picked using various selection approaches, is shown in

Table 6. Identification Accuracy with different feature-selection methods and subsets.

Classifier	Corr. Coef.	Fisher Score			Info. Gain		RFE			All Feat
		Top	Top	Top	Over	Over	10	20	30
		10	20	30	0.1	0.15	Feat	Feat	Feat
ANN	0.77	0.59	0.82	0.84	0.83	0.8	0.79	0.84	0.84	0.85
RF	0.8	0.59	0.82	0.83	0.83	0.82	0.81	0.84	0.84	0.84
SVM	0.76	0.58	0.8	0.85	0.83	0.78	0.77	0.85	0.85	0.85
Voting	0.79	0.6	0.83	0.86	0.85	0.81	0.8	0.86	0.87	0.87
XGBoost	0.82	0.6	0.86	0.87	0.86	0.85	0.83	0.88	0.88	0.88

. In general, utilizing no feature selection approach, the identification Accuracy reported by all classifiers examined was higher. However, all classifiers showed Accuracy results when utilizing the RFE that were comparable to those achieved when the classifiers were applied to all features, especially when using subgroups of 20 or 30. Evidently, the Accuracy score increased as the number of features increased, demonstrating that the statistically extracted features contributed well to class distinction.

4.3. Classifiers’ Comparisons

A reasonably big number of metrics were used here in order to determine the best-possible classifier for the proposed identification system, as each has its own strengths and weaknesses. For example, the measures that are most-frequently used to evaluate the performance of classification algorithms are Accuracy, Precision, Sensitivity, and Specificity [58]. These metrics, however, are not enough for classifier evaluation in class-imbalanced learning and are susceptible to the data distribution [59]. On the other hand, the true negatives and classes’ contributions to overall performance are not taken into consideration by the GEO and F1 [45], however, because they are less influenced by the majority classes in the imbalanced data, and they are beneficial evaluation measures for the performance of different machine learning methods [60]. The IBA metric combines a measure of how dominating the class with the highest individual Accuracy rate is with an unbiased index of its Overall Accuracy [45]. Previous studies indicated that all performance metrics, with the exception of the AUC, were weakened by skewed data distributions [61,62,63]. For these reasons, the AUC in addition to the aforementioned metrics were selected to gain a better insight into the performance of the proposed identification system using a specific classifier.

The highlighted values in

Table 7. Classifiers’ performance with different metrics. Bold values represent the top level of performance.

Classifier	Pre	Rec	Spe	F1	GEO	IBA	AUC	Acc
ANN	0.85	0.85	0.98	0.85	0.91	0.82	0.99	0.85
DT	0.75	0.75	0.97	0.75	0.85	0.71	0.86	0.75
GNB	0.5	0.5	0.94	0.46	0.65	0.44	0.87	0.5
KNN	0.83	0.83	0.98	0.83	0.90	0.8	0.96	0.83
LR	0.63	0.63	0.95	0.62	0.77	0.58	0.93	0.63
RF	0.84	0.84	0.98	0.84	0.91	0.81	0.99	0.84
SGD	0.55	0.54	0.94	0.53	0.71	0.49	0.90	0.54
SVM	0.85	0.85	0.98	0.85	0.91	0.82	0.99	0.85
Voting	0.87	0.87	0.98	0.87	0.92	0.84	0.99	0.88
XGBoost	0.88	0.88	0.99	0.88	0.93	0.86	0.99	0.88
LSTM	0.89	0.89	0.99	0.88	0.93	0.86	0.99	0.89

showed the best results of different metrics obtained after applying all classifiers employed to identify 10 users based on their accelerometer signals obtained from the original HMOG dataset without filtering. The LSTM deep learning model had the best performance, scoring an 89% Overall Accuracy, 89% Precision, 89% Recall, 99% Specificity, 88% F1-Score, 93% GEO, 86% IBA, and 99% AUC. The XGBoost classifier came in second, scoring an 88% Overall Accuracy, 88% Precision, 88% Recall, 99% Specificity, 88% F1-Score, 93% GEO, 86% IBA, and 99% AUC.

Given that HMOG is a class-imbalanced dataset, XGBoost’s successful performance may be explained by its capacity to modify the training process to place a greater emphasis on the misclassification of the minority class in such datasets [49,56]. Based on the results shown in Table 7, apparently, LSTM was the best choice for the proposed identification system, because it effectively addresses the vanishing gradient and long-term dependency issues and concentrates on the pertinent time series data [64,65]. Figure 9 shows the Accuracy and Loss curves during the training of the LSTM model on 10 users.

It is important to note that all machine learning methods tested in all experiments used a five-fold cross-validation approach with their own default parameters.

Almost similar results were obtained when increasing the number of users to 20, 30, and 40. Figure 10 shows that the AUC maintained its value despite the increase in class numbers. The F1-Score and GEO were also found to be less affected by the number of classes since they were computed for each class and then averaged over all classes, so they were less influenced by the number of classes than, for example, the Accuracy and other metrics, as shown in Figure 11 and Figure 12, which illustrate the performance of the two top classifiers, XGBoost and LSTM, respectively, using four metrics (Accuracy, F1-Score, GEO, and AUC) for identifying varying numbers of users (10, 20, 30, 40 and 50).

4.4. Authentication

The procedures of identification and authentication are inextricably linked since they both entail recognizing and verifying the legitimacy of users. However, identification is more complex than authentication because the feature space includes features from numerous subjects, but authentication is limited to a single subject, i.e. learning binary class data is easier than learning multi-class data [66].

In order to supplement this investigation, many experiments on authenticating a mobile phone user using accelerometer and/or gyroscope data were conducted. For this purpose, two user profiles were generated using data from 10 randomly selected users. The initial profile, called User1, was made up of data from a single user chosen at random who represented the positive class with 3027 examples, while the other nine users represented the imposters or negative class with 23,340 examples. The second profile, User2, had data from nine other users who made up the negative class or imposters and another random user who represented the positive class with 2559 samples.

The authentication experiments were conducted on data obtained from three sources: the accelerometer, gyroscope, and a combination of both. We omitted the data acquired from the magnetometer because magnetometers are impacted by environmental magnetic variation and other factors [67].

The same aforementioned features were employed, but this time, the deep learning LSTM and XGBoost were applied only, because they were the highest performers, as indicated by the previous identification results.

Table 8. The authentication performance of XGBoost on data obtained from different data sources. Bold values represent the top level of performance.

Sensor	User1							User2
	Pre	Rec	Spe	F1	GEO	IBA	Acc	Pre	Rec	Spe	F1	GEO	IBA	Acc
Accel	0.97	0.76	0.99	0.85	0.87	0.74	0.97	0.93	0.71	0.99	0.80	0.84	0.68	0.96
Gyro	0.94	0.65	0.99	0.77	0.80	0.62	0.95	0.88	0.32	0.99	0.47	0.56	0.30	0.93
Accelerometer + Gyro	0.98	0.83	0.99	0.90	0.91	0.82	0.98	0.95	0.80	0.99	0.87	0.89	0.78	0.97

and

Table 9. The authentication performance of LSTM on data obtained from different data sources. Bold values represent the top level of performance.

Sensor	User1							User2
	Pre	Rec	Spe	F1	GEO	IBA	Acc	Pre	Rec	Spe	F1	GEO	IBA	Acc
Accel	0.98	0.97	0.99	0.98	0.99	0.99	0.99	0.96	0.95	0.99	0.95	0.97	0.94	0.99
Gyro	0.95	0.87	0.99	0.91	0.93	0.86	0.98	0.88	0.72	0.99	0.79	0.84	0.69	0.96
Accelerometer + Gyro	0.99	0.99	0.99	0.99	0.99	0.99	0.99	0.99	0.99	0.99	0.99	0.99	0.99	0.99

present the five-fold cross-validation results of both classifiers employed for smartphone user authentication.

As shown in Table 8 and Table 9, it is clear that the LSTM method performed better than the XGBoost approach in binary class problems across a range of data sources and metrics. This achievement is owed to LSTM’s capacity to learn long-term correlations in sequences, especially in time series data and its capacity to resolve the vanishing gradient problem [68].

The confusion matrices illustrated in Figure 13 demonstrate the importance of combining gyroscope and accelerometer data. This demonstrates that, when the two sensors were utilized in conjunction, practically perfect outcomes were obtained, particularly when using LSTM. However, when considering only the gyroscope data, it did not perform as well as the accelerometer on different metrics and in different settings. This observation provides a strong rationale for using solely accelerometer data for identification in the earlier sections.

4.5. Comparison to State-of-the-Art Methods

Table 10. Comparison of the results with those of some related works.

Work	Sensors	Dataset	Approach	Metrics
[26]	Accelerometer Gyroscope Gravity	HMOG, Proprietary Dataset	HMM-UBM	EER 14.8%, EER 19.2%
[27]	Gyroscope Magnetometer	HMOG Tab and Keystroke	SVM	HMOG, Tap and Keystroke EER 7.16%
[47]	Gyroscope Scroll	HMOG	Deep Learning	EER 14% Acc 88% Pre 80%
[69]	Keystroke	HMOG	RF	Acc 68% Pre 71% Rec 76% F1 73 % AUC 72% MCC 59%
This Study/ Identification	Accelerometer	HMOG	LSTM	Pre 89% Rec 89% Spe 99% F1 88% GEO 93% IBA 86% AUC 99% Acc 89%
This Study/ Authentication	Accelerometer Gyroscope	HMOG	LSTM	Pre 99.9% Rec 99.9% Spe 99.9% F1 99.9% GEO 99.9% IBA 99.9% AUC 99.9% Acc 99.9%

shows the comparison to some of the state-of-the-art methods that used the HMOG dataset. The used sensors, the datasets, the approach, and the evaluation metrics are shown in this table. As mentioned before in the Related Work Section, none of the earlier studies evaluated their models with important metrics such as the IBA, AUC, and Specificity which are important to evaluate models’ performances.

As shown in Table 10, the proposed identification system using LSTM based on the aforementioned statistical features obtained from the smartphone accelerometer signal outperformed existing techniques, which were primarily developed for the authentication of a user (a binary classification problem), whereas the proposed system was created for the identification of multiple users (a multi-class classification problem), which is a more-challenging problem in machine learning, as supported by García and Ortiz [70], who argued that many classifiers either perform better with binary classification problems or are created expressly for them, which is a problem in the general case of classification. This is also evident from the authentication results of this study, where perfect user authentication was achieved, particularly when combining the gyroscope and accelerometer data using LSTM.

4.6. Research Contributions

The novelty and contributions of this work can be summarized as follows:

• Investigation of utility: The article investigated the potential applications and usability of accelerometer data, with a focus on user identification and authentication. The study intended to provide insights into the practical application of accelerometer data by using machine learning techniques and employing the preprocessing and feature selection approaches.
• User identification: To reliably identify users based on accelerometer data, the article employed a variety of machine learning algorithms, including deep learning algorithms, standard classifiers, and voting classifiers. For training and analysis, the study made use of the HMOG public dataset.
• Data preprocessing and feature selection: The research analyzed several preprocessing approaches and feature-selection strategies to increase the performance of the machine learning algorithms. These technologies seek to clean and enhance accelerometer data in order to improve the Accuracy of user identification.
• User authentication: The study extended the initial scope of user identification to cover the authentication process. It investigated the use of gyroscope data in conjunction with accelerometer data to examine their combined impact on authentication Accuracy.
• Sensor comparison and analysis: The study examined the efficacy of accelerometer data alone for authentication and evaluated the impact of gyroscope data on the authentication process. This comparative analysis delved into the individual and combined contributions of these sensors, demonstrating that accelerometer data are superior to gyroscope data for authentication and that combining both is the best option, omitting the magnetometer’s data, as they are impacted by the environment.

Overall, the research contributes to a better understanding of the utility of smartphone sensors’ data, particularly in user identification and authentication. It provides significant insights into the potential applications and performance optimization approaches associated with these fields.

5. Conclusions

This paper investigated the applicability of smartphone accelerometer data in identifying users and proposed the best components that can be used for a complete application. To this end, several machine learning methods were used to learn the accelerometer data of each user in order to identify them. These included a few chosen techniques from deep learning, traditional classifiers, and voting classifiers, all of which were trained using the users’ accelerometer signals obtained from the HMOG public dataset [22]. Moreover, the use of some preprocessing methods to clean the data was investigated, in addition to investigating some feature-selection methods to improve the performance of the identification system.

The large set of experiments conducted for the purpose of this study yielded the following conclusions:

The best performance in user recognition was demonstrated by the LSTM deep learning system, followed by XGBoost. These approaches were successful in properly identifying users based solely on accelerometer data.

Using bandpass filtering resulted in poor identification. Hence, it is better to avoid using such filtering in this context because it results in information loss.

In terms of feature selection, the RFE method beat the other methods investigated. It was notably beneficial in improving the identification results when used in conjunction with LSTM. It should be noted, however, that all statistical features preserved their importance during the identification process.

Even when the number of users expanded from 10 to 50, the system consistently performed well in terms of identification. This robustness suggests that the system was capable of handling large-scale user identification scenarios.

Furthermore, in terms of user authentication, this study found that the accelerometer data outperformed the gyroscope data and that using LSTM when combining both signals resulted in almost perfect user authentication.

However, the limitation of this study included the following:

• Out of the three sensors, this study only used the data from the accelerometer for identification and the accelerometer and gyroscope for authentication, leaving out the information obtained from the magnetometer, which might be beneficial to the identification process, even though it is shown to be affected by the environment and other factors [67].
• The signal’s magnitude and FFT of the magnitude were both used to extract the statistical features. It is necessary to investigate additional methods for reducing the dimensionality of time series, such as wavelet and discrete cosine transformations.
• Only one deep learning method (LSTM) was used in this study; more recent deep learning approaches need to be investigated such as the gated recurrent unit [71] and quasi-recurrent neural networks [72].

These limitations will be the focus of our future research so as to be able to devise more-accurate identification/authentication systems. Future studies should also look into incorporating advanced optimization methods within the proposed framework. These algorithms, which include hybrid heuristics, metaheuristics, adaptive algorithms, self-adaptive algorithms, and island algorithms, have been shown to be useful in addressing difficult decision problems in a variety of disciplines such as online learning [73], scheduling [74,75], multi-objective optimization [76], transportation [77], medical data, document classification, and other related fields [78]. We can demonstrate the versatility of advanced optimization algorithms and how they might improve decision-making processes in numerous situations, including the specific decision problem addressed in our study, by investigating their use in these diverse fields, which may contribute to the advancement of research in this field and open avenues for further exploration and innovation.