1. Introduction
Pedestrian dead reckoning (PDR) is a widely-used pedestrian tracking solution in satellite signal-masking environments where global navigation satellite systems such as the global positioning system (GPS) cannot work correctly [
1,
2,
3,
4]. The cumulative error problem is the main obstacle for achieving high tracking accuracy in both the Inertial Navigation Systems (INSs) and the Step-and-Heading Systems (SHSs) of PDR [
5,
6,
7]. For example, SHSs adopt the step-and-heading strategy to calculate the step vectors (
step length, heading) from the inertial sensor data and then evaluate the pedestrian trajectory according to the step vectors in a step-by-step way. However, such recursive estimation accumulates large errors originated from sensing drift with the increase of walking distance. Researchers have devoted a lot of effort towards solving the problem. Their approaches can be generally divided into two categories: (1) mining latent information to correct sensor errors; (2) introducing auxiliary information to adjust deviated trajectory.
Latent information assisted methods utilized prior knowledge or general inference as an extra supplement to fuse inertia step vector estimates for improving pedestrian trajectory accuracy. In foot-mounted INSs, the zero-velocity update (ZUPT) algorithm is a workable solution to limit long-term accumulation of errors [
8,
9,
10,
11]. The ZUPT algorithm is based on the fact that the speed of the sole should be zero during the stance phase period, when the sole is completely in contact with the ground. As such, the acceleration integral could be avoided [
12]. The building layout is another source of latent information closely related to indoor positioning services, which can affect pedestrian motion characteristics. For instance, pedestrians tend to walk in straight lines in straight corridors [
13], or in one of several dominant directions of the buildings [
14,
15,
16]. Pedestrian trajectory and heading direction can be inferred with the help of such layout characteristics and geometries in buildings. However, those inferences are not general enough in many indoor scenarios. For example, a pedestrian could have walked not according to layout dominant directions for a long time [
17].
Auxiliary information assisted methods adopt location-specific information inherent in the environment or created by additional technologies to adjust the PDR positioning result via a fusion algorithm. The auxiliary techniques or information include Wi-Fi [
18,
19,
20,
21,
22,
23], Bluetooth [
24,
25], Ultra-Wideband (UWB) [
26,
27,
28], visible light [
17,
29], and indoor map [
30,
31,
32,
33,
34,
35]. Due to the azimuth effect, the procedure of information fusion and trajectory calibration is a nonlinear model that can be solved by particle filtering [
20,
28,
33,
34] and extended Kalman filter (EKF) [
36,
37,
38]. The auxiliary information-assisted methods are more applicable in practice, since they do not depend on a specific building layout. However, most of the auxiliaries require additional hardware infrastructure, which often has expensive deployment costs.
With miniaturization, lightweight, and low power consumption of sensors, more sensors can be integrated into mobile terminals and wearable devices. For such considerations, we have initialized experiments of using a lightweight and portable 24 GHz
frequency-modulation continuous-wave radar (FMCW radar, simplified as radar hereafter) to measure the distance between a radar and a wall.
Figure 1 presents our experiment device and experiment scenario, where a student used a stand to connect an external yet small radar with a smartphone. On the one hand, we recognized that such wall distance estimations can be exploited to improve the accuracy of inertial sensor-based PDR. On the other hand, although a radar can estimate the distance to a wall, the estimation could fluctuate severely during pedestrian walking due to the irregular indoor layout, such as doors, windows, corners, obstructions, etc. Therefore, we need to first calibrate wall distance estimations from a radar and next fuse such estimations to adjust step positions.
In this paper, we propose a Radar-assisted Pedestrian Dead Reckoning scheme, called RadarPDR, for indoor tracking, which employs an external radar for trajectory adjustment. To the best our knowledge, we are the first to use and experiment with an FMCW radar for indoor pedestrian tracking. In RadarPDR, we first propose a segmented wall distance calibration model, which exploits the space-time correlation of neighbor points to deal with the radar ranging noise caused by irregular indoor building layouts. We next design a hierarchical particle filter together with an extended Kalman filter to fuse calibrated wall distances with acceleration, angular velocity, and azimuth information to adjust pedestrian step positions. We conduct field experiments in typical indoor scenarios. Results validate the effectiveness of our RadarPDR in terms of lower tracking errors compared with the widely used inertial sensor-based PDR scheme. Our main contributions can be summarized as follows:
Employ an FMCW ranging radar to assist inertial PDR for indoor pedestrian tracking;
Correct distance measurement errors caused by concave/convex wall structure in actual indoor environments;
Fuse corrected wall distance, pedestrian step length, and head direction for pedestrian trajectory adjustment.
2. Related Works
Radio-frequency (RF) is the most common auxiliary technology, such as Wi-Fi and Bluetooth, to assist PDR indoor tracking. Some research has proposed to integrate fingerprinting-based localization with PDR [
18,
19,
20,
21,
39]. For example, Shi et al. [
19] use a weighted K-Nearest Neighbor algorithm to obtain Wi-Fi RSS (received signal strength) fingerprinting locations, which are fused with PDR locations via a simple weighted average. Some researchers propose to fuse the lidar data and inertial sensor data to solve the accumulative error problem in localization [
23,
39,
40,
41]. Liu et al. [
39] develop a radio receiver to estimate the Doppler speed and range information from digital terrestrial multimedia broadcasting (DTMB) signals and carry out an extended Kalman filter (EKF) algorithm to fuse the information for the PDR. Li et al. [
20] propose to construct a radio map from pedestrian trajectories for RSS fingerprinting, while adjusting trajectory with the aid of Wi-Fi RSS fingerprinting results via selective particle filtering. These methods need to construct a radio map for fingerprnting either by site survey or crowdsourced trajectories.
Some research fuses the trilateral localization method with PDR [
22,
33,
39,
40,
42,
43,
44]. Treating iBeacon as an auxiliary, Xia et al. [
33] calculate distances between pedestrian and bluetooth anchor nodes based on an RSS path loss model to estimate pedestrian locations by the trilateration method. These locations are fused with PDR results via a particle filter. Tiwari and kumar Jain [
22] propose a Heron-bilateration-based position determination (HBPD) technique to locate a mobile device with a Wi-Fi
access point (AP). They also propose a hybrid indoor localization system (HILS) to integrate the HBPD technique with the PDR method. These methods require enough anchor points with known locations. Xu et al. [
34] propose an enhanced particle filter that fuses smartphone built-in sensors’ outputs and the distances in between smartphones. Tao et al. [
42] continuously collect light intensity values from smartphones, which are used to detect light source location for PDR correction. De Cock et al. [
43] present a smartphone-based multi-floor indoor localization system and Asraf et al. [
45] propose a deep-learning PDR framework based on smartphones’ inertial sensors. Tian et al. [
28] propose a PF-based fusion algorithm to correct position error due to sensor drift in a PDR-only system. Compared to the path loss model, these methods have higher distance estimation accuracy. However, almost all RF-based auxiliary technologies require pre-deployed hardware infrastructures.
Indoor layout is another piece of frequently used auxiliary information, mainly working in two ways: deterministic trajectory matching [
30,
31,
32] and probabilistic map model [
33,
34,
35]. Lan and Shih [
31] correct the drift error of the gyroscope by comparing the geometric similarity between a PDR trajectory and an indoor map. Nguyen-Huu et al. [
32] detect a potential landmark based on turning behaviors in corners or door areas and then update the current position with the landmark position for meeting some physical constraints. Zhou et al. [
40] use the conditional random field (CRF) model to estimate the optimal landmark sequence passed by pedestrians with the help of a pseudo indoor plan for PDR position correction. On the other hand, probabilistic map models usually occur in PF-based fusion schemes [
33,
34]. In these methods, an indoor environment is firstly divided into reachable and unreachable areas, which serve as particle weights in the particle filtering process. For example, Koroglu et al. [
35] use a continuous probability distribution function to convert a floor plan into a likelihood heat map for weighting particles. However, these methods require a prior knowledge of indoor layouts.
For ranging-assisted indoor PDR, many research studies calculate distances by communicating with surrounding Wi-Fi access points and then integrate the distance measurements with inertial data to improve positioning accuracy [
34,
46,
47,
48,
49]. For example, Sun et al. [
46] perform ranging based on Wi-Fi fine timing measurement technology and further calculate position through their improved trilateral positioning algorithm. The ranging-based positions are eventually fused with inertial PDR results under an EKF framework. Other research attempts to integrate active ranging devices. Sadruddin et al. [
50] integrate inertial PDR as an odometry into a body-mounted laser scanner platform for simultaneous localization and mapping, which also enables more accurate pedestrian localization. Mao et al. [
51] use a smartphone to measure distances to a surrounding wall. Wu et al. [
48] recognize the steady-heading and non-steady-heading human activities to optimize the PDR without any prior knowledge and Zhang et al. [
47] use the Wavelet-CNN deep learning network to train the smartphones’ built-in MEMS sensor data for human activity recognition.
This paper utilizes an external FMCW radar to assist PDR tracking. Since such a radar is entirely a self-transmitting and self-receiving device, our RadarPDR scheme can be applied to environments without prior indoor layouts and does not need additional expensive hardware infrastructure.
3. Problem Description
We investigate how to use a ranging radar to adjust pedestrian trajectory for indoor tracking. As illustrated in
Figure 2, a user holds a smartphone with an FMCW radar, and a sequence of steps represents his trajectory. Let
denote the pedestrian pose at the
n-th step, where
is the heading orientation and
the step coordinate. The radar is used to measure the distance between the user and a wall. The sampling rate of radar is 10 Hz, which is higher than the pedestrian walking speed. Thus, between two steps, we can have
M wall distance measurements, denoted by
. Each observation
links an observation point and a wall point. The observation point is the pedestrian pose when this observation occurs. We assume that
M observation points are evenly distributed within one step. The wall point is a virtual point on the wall surface, and the distance between the two points is
.
The proposed RadarPDR adopts a particle filtering framework for trajectory adjustment. Although the principle is straightforward, its implementation faces many practical challenges: From our field experiences, if a user walks strictly in parallel with a flat wall, distance measurements of an FMCW radar have high accuracy, with ranging errors often smaller than 0.2 m. However, such ideal cases are not often in practical indoor environments. On the one hand, a user may not always walk along with a strict line, and some turning steps are not unexpected. Sometimes, other people may pass across the user and a wall, leading to interferences in distance measurements. On the other hand, it is not easy to find a long flat wall in many buildings, as indoor walls often have some concave and convex structures, such as recessing doorways or protruding pillars. Furthermore, when passing by a corner, wall distance measurements have even larger fluctuations, which often leads to poor positional information for failed trajectory adjustment.
Figure 2 plots the distance measurements of a trajectory in our field experiments. On the one hand, it can be seen that irregular wall structures can cause some distance pulses, and wall corners can cause large distance variations. On the other hand, it can be seen that, besides such distance pulses and large variations, some distance measurements can remain relatively stable in a short walking interval. These observations inspire us to design an adaptive scheme to enforce trajectory adjustment: That is, the pedestrian pose
is subject to adjustment only when the wall distance
contains sufficient positional information. Otherwise, we accept the pose estimation from the underlying dead reckoning system. Now, the research questions can be summarized as follows:
When to enforce pedestrian pose adjustment?
How to correct measurement for wall distance ?
How to adjust a pedestrian pose based on ?
We next present our solution RadarPDR to these questions.
4. The Proposed RadarPDR Scheme
4.1. Overview
Figure 3 presents the system diagram of our RadarPDR scheme, which consists of four main modules: (i) data sensing and preprocessing; (ii) inertial PDR, (iii) wall model calibration; (iv) multi-source information fusion.
As sources, accelerometer, gyroscope, and radar continuously yield raw observation data according to their acquisition frequency. The data preprocessing module cuts these data streams into step-wise pieces implemented by peak detection [
52] based on acceleration observations. Once a new step is completed, these sorts of observations will be distributed into the following processing modules.
Following standard procedures, the inertial PDR module works to estimate step length and heading increment using inertial sensor observations. Step length is estimated by the Weinberg algorithm [
53] based on the acceleration series; heading increment is computed based on the azimuth observations from the gyroscope-based orientation sensor embedded in our Android smartphone. The wall model calibration module aims to tackle the corner and barrier problems per our discussion in the previous section. In contrast, the multi-source information fusion module tries to integrate all available information and finally estimates an accurate pedestrian trajectory.
4.2. Wall Model Calibration
This module focuses on two realistic challenges in the wall model: the corner problem and the barrier problem. We assign an operation indicator for each step based on corner detection, which works to tackle the adjustment failure in the corner zone by an adaptive operation. Moreover, a wall distance correction algorithm is developed to handle the outlier issue due to walls’ barrier structures.
4.2.1. Operation Indicator Assignment
The proposed scheme is based on a flat-wall observation that the wall is approximately flat during several adjacent steps. A wall parameter tuple
with slope
the slope and
the intercept can represent the wall model. Long trajectories can be related to segmented flat walls with several corners. For example,
Figure 2 depicts a trajectory beside a wall consisting of two flat-segments and one corner. In the non-corner zone, each flat-segment needs independent parameter initialization and an updating procedure. In addition, wall distances obtained in the corner zone show significant fluctuation. In short, a corner and non-corner zone need different operations in the following modules. Therefore, we generate an operation indicator for every step based on corner detection. In RadarPDR, each step belongs to one of three states: operate, suspend, and reset. The operation of each state is as follows: (i) trajectory adjustment and wall parameter updating; (ii) dead reckoning; (iii) dead reckoning and wall parameter initialization.
The indicator assignment principle is as follows: Firstly, if the heading increment
is more significant than an angle threshold
, then
n-step is judged to be in a corner zone, and its corresponding indicator
is assigned to
suspend; otherwise we think that
n-step is in a flat-segment zone, and it needs further judgment. Then, if
n-step belongs to the beginning
steps of the flat-segment,
is assigned to
reset; otherwise,
is set to
operate.
Figure 4 presents wall distance observations, heading orientations, and corresponding indicators of several steps around a corner, where
and
=
. The
is chosen according to our experiences and has been proven practicable in our experiments to detect whether a pedestrian has cornered.
4.2.2. Wall Distance Correction
In this paper, barrier refers to the convex and concave structures embedded in a flat wall, such as recessing doorways or protruding pillars. These unexpected barriers violate the flat-wall model, yielding noises in wall observations.
Figure 5 depicts a series of noisy wall distance data measured in a field experiment. We classify these noises into two categories: (i) random noise; (ii) impulse noise. The random noise comes from inherent radar ranging error, just like many other sensors. It has a relatively small amplitude; the impulse noises come from internal signal fluctuation or wall barrier structures, presenting a much larger amplitude than random noise. Our distance correction algorithm discussed in this subsection aims to handle both random and impulse noises to ensure that the flat-wall model is well satisfied.
Firstly, we discuss the random noise, which can be approximated to Gaussian white noise. To simplify the description, we assume the number of single-step observations and simplify the notation of the single observation at the n-th step to . The general situation () will be explained later. Let and denote the change rate of and the process noise. Let and denote the wall distance observation and the random noise following a Gaussian distribution with zero mean and variance, respectively. The random noise can be corrected by a Kalman filter (KF) with the following state-space model:
The prediction functions describe the state-transition procedure of wall distance and its change rate, and they are designed by the fact that pedestrians tend to walk in a straight line or change their directions slowly in most cases. When a pedestrian walks in a straight line, the wall distance change rate remains unchanged, while the process noise gives a fuzzy range for the direction changing case.
Considering the impulse noise denoted by
, the above observation function is rewritten as
Due to the lack of precise statistics of , the impulse noise cannot be eliminated easily by general Kalman filtering as the random noise is. Thus, we design a two-stage correction strategy. That is, for each observation , we judge whether it is an impulse noise firstly, and then select the correction method correspondingly.
The judgment stage is based on a threshold method imposed on the wall distance change rate of
. On the one hand, a normal walking trajectory can be regarded as a continuous curve with a relatively large curvature radius, usually a few meters, except in the corner zone, which leads to relatively small
in inlier observations. On the other hand, a much larger amplitude of impulse noise or outlier observation makes the wall distance change rate estimation
severely deviate its normal range. In short, there is a huge margin between the change rate of inlier wall distance and that of the outlier one, as shown in
Figure 6. Therefore, we define the outlier detection criterion as
where
is the upper limit of the change rate of the inlier wall distance selected in the margin.
The correction stage adopts the KF solution with the state-space model given by Equations (1)–(3), if there is no outlier. Otherwise, it will switch to a modified KF version. In our problem, a KF’s observation is much less reliable than its prediction when an outlier occurs. The prediction is close to the ground truth in a short time due to the slow-change feature of wall distance change rate. Thus, we drop the noisy observation directly and use modified prediction functions to obtain wall distance correction. The modification lies in the fact that the trend instead of the single estimation of
is used here. The final correction is summarized as
where
denotes the trend of a few recent distances at the
n-th step, calculated by an exponential average of wall distance change rate corrections.
The proposed wall distance correction algorithm in a general case is summarized in Algorithm 1. Note that the judgment-correction procedure is just simply repeated
M times according to the order of occurring, when we have
M observations in one step.
Algorithm 1 Wall distance correction algorithm |
Input: : wall distance observations Output: : wall distance corrections
for each step do for each observation do 1. Get state estimations by standard Kalman filtering: , = KF () 2. Detect outlier and compute correction: if then , else , end if 3. Update trend: end for end for |
4.3. Multi-Source Information Fusion
After processing by the upper modules, for each step, we now have information including step length
, heading increment
and wall distance
. We next fuse them to estimate pedestrian pose
. Based on the Bayes filtering framework, we define a motion model and an observation model to output their relation. The motion model presents the recurrence relation of pedestrian pose
with step length
and heading increment
as follows:
As shown in
Figure 7, given the pedestrian pose
, the ranging direction
and the wall distance
, the wall point
is represented as
Combining the wall model
and the wall point
,
is represented as
Note that the ranging direction is always perpendicular to the pedestrian heading in our device setup. To replace
with the heading direction
under the relationship
,
is then represented as
In short, the observation model gives wall distance
at observation point
with wall parameters
as follows:
Now, it can be summarized that , , are the given information inherited from the upper modules, while or , are the unknown states to be estimated in this module (note that ). Furthermore, in the observation model, if serves as the state to be estimated, then needs to be the known parameter and vice versa. Therefore, we adopt a PF-EKF architecture to solve this dual-estimation problem, where PF takes charge of trajectory adjustment for , while EKF is in charge of wall parameter estimation for .
4.3.1. Trajectory Adjustment
The basic idea of PF-based trajectory adjustment [
54] is as follows: During each round of iteration, PF makes a group of guesses, known as particles, around the unreliable state estimated by the motion model. This is the particle sampling procedure. Some of the particles are positive adjustments, closer to the ground truth, while others are negative since they have large deviations. We propose a judgment procedure to weight the particles. Finally, another sampling procedure is implemented on the weighted particles. Those with high weights are more likely to be selected, leading to an improvement in pedestrians’ pose estimation. We next explain these three steps in details below.
The sampling procedure is performed on step length
and heading increment
from which the errors of the pose originate. We assume that the errors of
and
follow Gaussian distribution with zero mean and
standard deviation, respectively:
Then, the corrections are represented by
where
n is the step index and
k is the particle index. According to the motion model, particle pose is reckoned as
K particles are generated within one step by repeating the above procedure.
The essence of weight calculation is to judge the reliability of particles. Each particle can obtain its wall distance prediction
by feeding its observation point
and wall parameters to the observation model. At the same time, a distance observation
obtained by radar is available. By calculating the proximity between
and
, we can obtain the particle weight
. This procedure is formulated as
where
is the variance of distance observation residual
at the
n-step and
is the wall parameter, and both of them are calculated in the EKF part. The weight
considers the distance observation residual following a Gaussian distribution with zero mean and
variance, a larger residual meaning a lower weight. When we have
M observations in the
n-th step, we will obtain
M weights for one particle. Then, the weight of the
k-th particle at the
n-th step is calculated by an average of these
M weights:
Resampling is a two-sided procedure which aims to ensure that higher weights get resampled more while maintaining particle diversity. We adopt the systematic resampling algorithm introduced in [
55]. After the resampling procedure, the final estimated pose state is obtained by averaging the resampled particles.
4.3.2. Wall Parameters Estimation
Due to the absence of a prior map information, the wall parameters necessary for particle weighting are unknown to us initially. The proposed wall parameter estimation algorithm consists of two phases: the initialization phase and the updating phase. The initialization phase happens at the beginning steps of each flat-segment, while the updating phase happens at the remaining steps of that flat-segment.
We first introduce the initialization phase. Although the EKF algorithm is insensitive to initial values, wall parameters from random initialization may deviate from the ground truth a lot, which could distort the PF weighting procedure. To deal with such issues, we design an initialization strategy based on the flat-wall discovery as follows: Given an observation point
and wall distance observation
, its corresponding wall point
can be calculated by the following geometric relationship:
The wall parameters are estimated by using the least-square method with all wall points within this phase. In this phase, pedestrian poses and observation points are estimated by dead reckoning via the motion model, since the PF-EKF architecture does not work without wall parameters.
We realize a step-by-step online estimation for wall parameters via extended Kalman filtering with pedestrian pose and wall distance observations during the updating phase. Let
denote the wall parameters at the
n-th step. According to the flat-wall model,
changes little in several neighboring steps. We thus have the following state transition function:
where
is a two-dimensional identity matrix. The observation function is exactly the observation model given by Equation (
12), rewritten as
Note that here is considered as a known parameter from the PF. and are independent Gaussian noise processes with zero mean and and covariance matrices, respectively.
The extended KF utilizes only the estimated state
of the previous step and the current observation
to estimate the current state
, and the update procedures at the
n-th step are formulated below:
where
is the distance observation residual.
is the observation matrix that is equal to the Jacobi matrix of
,
where
.
I is an identity matrix. Just like the processing style in wall distance correction, we repeat the update procedure
M times according to the occurring order of these
M wall distance observations at one step.
In the PF part, wall parameters must ensure pedestrian poses, while this is the exact opposite in the EKF part. Consequently, these two parts are concatenated to boost each other, as illustrated in
Figure 8. At the iteration of
n-step, the step length, heading increment, wall distance observations from previous modules and wall parameters, and residual covariance from the EKF are fed to the PF, which outputs pedestrian pose estimation after the three procedures of new particles’ sampling, weighting, and resampling. Then, the pose estimated by PF and wall distance observations are fed into the EKF, which outputs updated wall parameters. The complete trajectory adjustment algorithm is summarized in Algorithm 2.
Algorithm 2 Trajectory adjustment algorithm |
Input: : step lengths, : heading increments, : wall distance,
: indicators Output: : pose estimations
1: for each step do
2: if the indicator is operate then
3: for each particle do
4: Get its pose according to Equations (13)–(15)
5: Get its weight according to Equations (16)–(18)
6: end for
7: Perform Resampling
8: Get pose estimation by average of resampled particles
9: Update wall parameters according to Equations (21)–(29)
10: end if
11: if the indicator is suspend then
12: Get pose estimation by dead reckoning motion model
13: end if
14: if the indicator is reset then
15: Get pose estimation by dead reckoning motion model
16: Initialize wall parameters
17: end if
18: end for |