Impact Analysis of Time Synchronization Error in Airborne Target Tracking Using a Heterogeneous Sensor Network

Abstract

This paper investigates the influence of time synchronization on sensor fusion and target tracking. As a benchmark, we design a target tracking system based on track-to-track fusion architecture. Heterogeneous sensors detect targets and transmit measurements through a communication network, while local tracking and track fusion are performed in the fusion center to integrate measurements from these sensors into a fused track. The time synchronization error is mathematically modeled, and local time is biased from the reference clock during the holdover phase. The influence of the time synchronization error on target tracking system components such as local association, filtering, and track fusion is discussed. The results demonstrate that an increase in the time synchronization error leads to deteriorating association and filtering performance. In addition, the results of the simulation study validate the impact of the time synchronization error on the sensor network.

1. Introduction

The use of unmanned aerial vehicles (UAVs) has attracted much attention in a broad range of applications. However, the ubiquitousness of UAVs in the airspace has introduced new risks arising from their potential misuse. For example, a drone intrusion disrupted Gatwick airport in December 2018, leading to a 33-h closure which resulted in an estimated loss of GBP 50 million [1]. These increased risks have necessitated the development of counter-unmanned aerial systems (CUAS) to identify UAVs, monitor their behaviors, and implement appropriate countermeasures. A CUAS typically deploys multiple heterogeneous sensors to accurately and promptly detect, identify, and collect target information for surveillance [2,3,4]. The distributed target data are then aggregated at the fusion center. In the sensor fusion process, a common notion of time across the sensors is essential, as correlations between the collected data are evaluated based on time information [5]. Therefore, tight synchronization across the heterogeneous sensor network is necessary to improve target tracking and sensor fusion.

Time synchronization aims to provide a common time shared between local sensors with unsynchronized clocks, thereby ensuring that all nodes in the sensor network are in alignment with a timing reference. Synchronization based on the global navigation satellite system (GNSS) is a typical solution, in which accurate time sources from satellites are used as the reference timing. In the context of time distribution, one-pulse-per-second (1-PPS) synchronizing signals from global positioning system (GPS) or GNSS receivers or synchronization protocols [6,7,8,9,10,11,12,13,14,15,16,17,18] allow local sensors to be aligned across the network through communication [17]. Although synchronization directly through GNSS clocks facilitates the realization of precise time accuracy, it requires additional equipment with access to the satellite signal from all local sensors. Time synchronization protocols are developed in order to distribute time over packet-switched networks and to synchronize distributed devices such as the network time protocol (NTP) [12] and precision time protocol (PTP) [13] for time-sensitive networking (TSN). Wireless TSN frameworks, such as reference broadcast synchronization (RBS) and time synchronization protocol for sensor networks (TPSN), have been developed as well [9,11,18]. However, GNSS-based synchronization relies on the GNSS system, and may be vulnerable to system failure, environmental interference such as urban canyons, and intentional disruptions such as jamming or spoofing with time error injection [7]. For example, in January 2021 GPS was noted to be unreliable within a 50-nautical-mile radius of the Denver International Airport, which caused disruptions to infrastructure and air traffic management applications. The incident lasted for 33 h, which led to drifting of the clocks for each subsystem during the disruption and caused the ground control station to become isolated [19]. To address concerns regarding GNSS reliability, it is necessary to use timing systems that do not directly rely on local GNSS receivers. An alternative is to use fault-tolerant clock synchronization based on redundant clock sources [8,9,10] or receiver-independent time synchronization [20].

Considering the importance of time synchronization in target tracking, it is essential to understand the potential correlation between time synchronization and target tracking performance. However, the existing studies have not extensively explored the effects of time synchronization. Typically, time synchronization and target tracking systems in sensor networks are designed independently. Previous studies have demonstrated that synchronization accuracy is influenced by several factors, including the quality of the clock source, the timestamp resolution, and the network topology. Additionally, characteristics of sensors such as radar, acoustic sensors, and cameras as well as their deployment in diverse environmental conditions, e.g., operations with various temperatures, and density or tracking systems applied to different domain such as underwater or aerial vehicles, can contribute to time synchronization errors [21]. A time synchronization protocol is typically established to meet a specified precision level, while target tracking systems are developed assuming that the time information is sufficiently accurate. Various filtering schemes aimed at improving state estimation in the presence of delayed measurements have been proposed [22,23,24], primarily focusing on state estimation within the target tracking system. In a target tracking system employing multiple sensors, data association and state estimation within the sensor fusion rely on temporal information from local sensors. In this regard, any imprecision in the time alignment among sensors can adversely affect these processes, leading to inconsistencies and compromised reliability in data fusion. Nevertheless, the influence of time synchronization errors on the target tracking system remains to be clarified. In light of the heightened demand for advanced tracking precision, particularly in the context of proximity operations for UAVs navigating dense urban environments, it is necessary to examine correlations between errors resulting from loose time synchronization and the overall performance of the target tracking system.

Therefore, the objective of this work is to investigate the impact of time synchronization errors on the performance of sensor fusion and target tracking. The target tracking system considered in this study consists of two system layers: a sensing layer using heterogeneous local sensors and a data processing and tracking layer located at the fusion center (FC). Similar to networked components, clocks are mathematically modeled in a linear form. The reference clock is accurate in the FC, while a time offset is assumed for each local clock to introduce errors to the timing in the network. Clock synchronization is assumed to occur when the time error between the local clock and the reference is within a predefined accuracy threshold, while time synchronization fails when the time error exceeds the threshold. Additionally, the influence of the time synchronization error on target tracking system components, such as local association, filtering, and track fusion, is discussed in order to clarify the correlation between the time error and the tracking performance. A simulation study is performed to validate the analytical results. Our findings indicate that the target tracking system performance can deteriorate due to increasing time synchronization errors. Notably, the system performance is evaluated against a number of target tracking performance metrics, including the root mean square error (RMSE), generalized optimal sub-pattern assignment (GOSPA) [25], and single integrated air picture (SIAP) [26].

The main contribution of this study lies in a systematic assessment of the influence of time synchronization errors on sensor fusion in target tracking systems. We design a sensor fusion system model that reflects time synchronization errors across sensor networks. Track-to-track fusion (T2TF) with time alignment in the local tracker is introduced to handle heterogeneous sensor characteristics. Moreover, we investigate the influence of time synchronization on individual components of the sensor fusion system, including filtering, association, and track fusion processes. Our analysis reveals observable performance degradation in terms of bias and error covariance. These findings suggest that the sensor fusion system experiences significant impairment when synchronization errors become non-negligible. Our simulation study provides empirical results illustrating the impact of synchronization errors on tracking performance. These insights can help to mitigate potential issues in target tracking systems that rely on sensor networks as well as in other network systems that require time synchronization [27,28].

The remainder of this paper is organized as follows: Section 2 details the time synchronization process and synchronization error considered in this study; Section 3 explains the target tracking system and sensor fusion technique; Section 4 discusses the influence of the time synchronization error on the target tracking system components; Section 5 presents the results of our numerical simulation; and Section 6 presents concluding remarks and recommendations for further work.

2. Preliminaries

This section introduces the time synchronization method using two-way communication and the time error model considered in this study. Figure 1 illustrates the network topology considered in this study, in which all sensor nodes are fully connected to a central node. The command and control (C2) serves as the central node; the server clock in C2 has access to a precision reference clock from a terrestrial source, serving as an alternate position, navigation, and timing (PNT) reference [10,17].

We assume that the server clock is accurate and serves as the reference clock $c_r\left(t\right)=t$. The local clock time can be expressed in the following first-order affine form [11]:

$$c_i\left(t\right)=t+{\Delta }_i\left(t\right),i=a,p$$

(1)

where the subscripts a and p represent active and passive sensor nodes, respectively, and ${\Delta }_i$ is the time error, which can be time-varying as follows:

$${\Delta }_i\left(t\right)={\alpha }_i(t-t_0)+{\theta }_{0,i}+{ϵ}_i$$

(2)

where ${\alpha }_i$ refers to the clock drift due to frequency mismatch, $t_0$ refers to the previous synchronization time instance, ${\theta }_{0,i}$ is the initial offset calculated at $t_0$, and ${ϵ}_i$ contributes to stochastic uncertainties, including unmodeled errors. The drift coefficient indicates the increasing time error over time. Considering a worst-case scenario, we assume that the drift for active sensor ${\alpha }_a$ is negative while that for passive sensor ${\alpha }_p$ is positive. To synchronize slave clocks to the server, the server clock time $c_r\left(t\right)$ is distributed to the slave clocks over a communication network and the local clock adjusts its offset as follows:

$$c_i\left(t\right)\leftarrow c_i\left(t\right)-{\hat{\Delta }}_i$$

(3)

where ${\hat{\Delta }}_i$ is the approximate offset to be determined. The synchronization accuracy determines the error magnitude. Several synchronization protocols standardize the accuracy requirement [12,13], and synchronization procedures have been developed to achieve the synchronization requirements over the networks [14,15,16]. However, the time synchronization performance in practical applications may be impaired by various factors, such as GNSS receiving failures or variable and asymmetric latencies [29]. To address this issue, viable failover solutions are required in order to ensure resilient reference timing distributions. In these instances, an accurate time source is distributed to the network server to ensure that time error is maintained within the required offset bound $\delta$. Similarly to the approach presented in [30,31,32], we conducted hardware experiments to obtain statistical results of time errors, for which the distribution is modeled as a normal distribution:

$${\Delta }_i\sim \mathbf{N}(m_{\theta },{\sigma }_{\theta }^2)$$

(4)

where $m_{\theta }$ and ${\sigma }_{\theta }$ are the mean and standard deviation, respectively, and are set to 422 μs and 62.293 μs following experimental results. The required offset bound is set to $max\left({\Delta }_i\left(t\right)\right)\le \delta =1$ ms for the worst-case analysis. In the worst-case scenario involving the largest time variation between local clocks, two synchronized clocks can drift from each other at a rate of at most $2{\alpha }_{max}$, where $\alpha \in [-{\alpha }_{max},{\alpha }_{max}]$. The time difference between node clocks is expressed as

$$|c_i\left(t\right)-c_j\left(t\right)|\le 2m_{\theta }+2{\alpha }_{max}(t-t_0).$$

(5)

To limit the relative offset to $\delta$, the maximal holdover interval ${\tau }_{sync}=t-t_0$ should be bounded as

$${\tau }_{sync}\le \frac{\delta -2m_{\theta }}{2\alpha }.$$

(6)

If the local clocks are not synchronized, clock skew increases the time difference from the reference value during holdover. If the holdover period is longer than (6), then the time synchronization error exceeds the requirement. The synchronization error due to longer holdover periods is discussed in Section 4.

3. Target Tracking System

Figure 2 illustrates the target tracking system structure explored in this study. The sensing layer exchanges data with the tracking and data fusion layer through network communication. The sensing layer consists of active and passive sensors. Sensors are distributed on the surveillance site to detect target signals. The target measurements are timestamped by the local clock and sent to the tracking and fusion layer through the communication network. The tracking and fusion layer possesses a main sensor fusion architecture based on T2TF, wherein a local tracker processes target measurements into local tracks which are then integrated into fused tracks via track fusion. Notably, T2TF enables more effective data transmission through network communication than the measurement-to-track structure, and as such has found wide application [33,34,35,36]. Local measurements are first processed at local trackers to estimate and predict tracks. The FC placed at the end of the data process aggregates the local tracks and fuses them into central tracks. The heterogeneous T2TF algorithm based on practical decomposition [37] helps to associate local tracks with different dimensions and synthesize them into a global track. The update rates in the tracking and data fusion layer and active sensor are synchronous, while the update rates in the passive sensor are two times smaller than those in other modules. The clock in the tracking and fusion layer is assumed to be synchronized to GNSS and can serve as a reference clock or server clock.

3.1. Sensor Model

In the sensing layer, two heterogeneous sensors are used, i.e., active radar and passive radio frequency (RF) sensors, in light of their widespread adoption in CUAS surveillance systems [4,38,39,40]. The radar sensor provides target position measurements, whereas the RF sensor passively detects target position only in two-dimensional space. The two sensors are asynchronous, with the passive sensors having a smaller update rate than the active sensor. The radar periodically scans a search space with sampling time $T_a=t_k-t_{k-1}$. In the k-th scan period, the radar may detect a target at $t_m\in [t_{k-1},t_k]$ and record the detected measurement timestamped using the current local time $c_a\left(t_m\right)$. The measurement in vector form can be expressed as follows:

$${\mathbf{z}}_a\left(t_m\right)=[r\left(c_a\left(t_m\right)\right),\phi \left(c_a\left(t_m\right)\right),\vartheta \left(c_a\left(t_m\right)\right)]\in {\mathbb{R}}^3$$

(7)

where $(r,\phi ,\vartheta )$ represents the target range, azimuth, and elevation angles expressed in the spherical coordinate system. Suppose that a target and the radar are located at $(x,y,z)$ and $(x_a,y_a,z_a)$, respectively; then, the measurement variables exhibit the following relationships:

$$\begin{array}{c}r=\sqrt{{(x-x_a)}^2+{(y-y_a)}^2+{(z-z_a)}^2},\hfill \\ \phi ={tan}^{-1}\left(\frac{y-y_a}{x-x_a}\right),\hfill \\ \vartheta ={sin}^{-1}\left(\frac{z-z_a}{\sqrt{{(x-x_a)}^2+{(y-y_a)}^2+{(z-z_a)}^2}}\right).\hfill \end{array}$$

(8)

The measurement is performed until the scanning is complete and then transmitted to the local tracker through the network at $t=t_k$, which inevitably introduces a time delay $\Delta t=t_k-t_m$.

In contrast, the RF sensor takes only the target measurement expressed in two-dimensional space. Let $T_p$ be the sampling time for the RF sensor, which is larger than that of the radar, i.e., $T_p>T_a$. The RF measurement can be expressed in vector form as follows:

$${\mathbf{z}}_p\left(t\right)=[r\left(c_p\left(t\right)\right),\phi \left(c_p\left(t\right)\right)].$$

(9)

If the RF sensor is located at $(x_p,y_p,z_p)$, then the measurements can be rewritten in terms of the relative positions with the target:

$$\begin{array}{c}r=\sqrt{{(x-x_p)}^2+{(y-y_p)}^2+{(z-z_p)}^2}\hfill \\ \phi ={tan}^{-1}\left(\frac{y-y_p}{x-x_p}\right).\hfill \end{array}$$

(10)

The detected target measurements are sent to each local tracker in the tracking and fusion layer through the network.

Table 1. Sensor specifications.

Sensors	Radar	RF
Detection update rate	1 Hz	0.5 Hz
Azimuth resolution (Deg)	3	5
Range resolution (m)	5	5
Elevation resolution (Deg)	3	-
Location (m)	(0,0,0)	(0,500,0)
Detection range (m)	1000	1000

summarizes the sensor specifications considered in this study.

3.2. Local Tracker

In the local tracker, sensor measurements received from local sensors are processed as local tracks. The state-space model is defined as

$$\begin{array}{cc}\hfill {\mathbf{x}}_i\left(t_k\right)& ={\mathbf{f}}_i\left({\mathbf{x}}_i\left(t_{k-1}\right)\right)+{\mathbf{w}}_i,\hfill \\ \hfill {\mathbf{z}}_i\left(t_k\right)& ={\mathbf{h}}_i\left({\mathbf{x}}_i\left(t_{k-1}\right)\right)+{\mathbf{v}}_i.\hfill \end{array}$$

(11)

Depending on the sensor measurement, the local tracker exhibits different state and measurement variables, i.e., ${\mathbf{x}}_a=[x,\dot{x},y,\dot{y},z,\dot{z}]\in {\mathbb{R}}^6$ and ${\mathbf{x}}_p=[x,\dot{x},y,\dot{y}]\in {\mathbb{R}}^4$, respectively. To simplify the notation, the current time step of the tracker is denoted by k and variables evaluated at $t_k$ are expressed with the subscript k. Each local tracker uses the association function to perform measurement-to-track association and performs filtering to estimate target states with time alignments of the measurement timestamp to the evaluation period. Association requires the discrimination of target measurements from clutter; the aim of the filtering process is to appropriately estimate the target states. At time step k, measurements taken at $t_m$ are timestamped as $c_i\left(t_m\right)$, which may lie in the track interval $[k-1,k]$. Therefore, it is necessary to perform time alignment in order to address the time mismatch between measurements and the track time calculated according to the reference clock.

This study adopts the standard extended Kalman filter (EKF) technique [41] to estimate local track states in the presence of asynchronous timing between the measurement and track. Specifically, we include an intermediate time index $k_0$ in the track interval $[k-1,k]$ to define two sub-intervals $[k-1,k_0]$ and $[k_0,k]$. Here, $k_0$ corresponds to the time instant at which the target measurement is obtained, i.e., $t_{k_0}=c_i\left(t_m\right)$.

3.2.1. Measurement-to-Track Association and Filtering

In the first period $[k-1,k_0]$, a posteriori state estimates and covariance matrix, i.e., ${\hat{\mathbf{x}}}_{k_0|k_0}$ and $P_{k_0|k_0}$ at $k_0$, respectively, are obtained using the standard EKF procedure. The target tracking algorithm in this period consists of a priori and measurement updates with measurement-to-track association. In the state prediction step, the track states in the previous step $k-1$ are predicted up to measurement update step $k_0$ to obtain a priori estimates. Let ${\hat{\mathbf{x}}}_{k-1}$ be the track estimates in the previous step. Using Equation (11), the a priori estimates and covariance matrix approximation using a linear model can be obtained as follows:

$$\begin{array}{cc}\hfill {\hat{\mathbf{x}}}_{k_0|k-1}& =\mathbf{f}\left({\hat{\mathbf{x}}}_{k_0|k-1}\right)\hfill \\ \hfill P_{k_0|k-1}& =F_{k_0,k-1}{P}_{k-1}{F}_{k_0,k-1}^T+Q_{k_0,k-1}\hfill \end{array}$$

(12)

where $F_{k_0,k-1}=\frac{\partial \mathbf{f}}{\partial x}{|}_{x_{k-1}}$ represents the Jacobian matrix of the state transition function and $Q_{k_0,k-1}$ is the process noise covariance matrix propagated from $k-1$ to $k_0$.

At measurement update step $k_0$, the standard EKF updates states and covariance matrix approximation are

$$\begin{array}{cc}\hfill {\hat{\mathbf{x}}}_{k_0|k_0}& ={\hat{\mathbf{x}}}_{k_0|k-1}+K_k\tilde{\mathbf{z}}\hfill \\ \hfill P_{k_0|k_0}& =P_{k_0|k-1}-K_k{S}_k{K}_k^T\hfill \end{array}$$

(13)

where $\tilde{z}$ denotes the innovation term for measurement updates, $K_k$ is the Kalman gain, and $S_k$ is the innovation matrix. In a cluttered environment, multiple measurements may be mixed with actual data from the target and incorrect data resulting from noise or false alarms; therefore, it is necessary to distinguish correct target measurements from spurious data. In this study, a probabilistic approach using a joint probability data association (JPDA) algorithm is used [42,43,44]. The predicted measurement and covariance of innovation are obtained as follows:

$$\begin{array}{cc}\hfill {\hat{\mathbf{z}}}_{k_0|k-1}& =H_k{\hat{\mathbf{x}}}_{k|k-1}\hfill \\ \hfill S_{k_0}& =H_k{P}_{k_0|k-1}{H}_k^T+R\hfill \end{array}$$

(14)

where $H_k=\frac{\partial \mathbf{h}}{\partial x}$ denotes the Jacobian matrix of the observation function. The association test between the predicted measurement and observations is performed by measuring the statistical distance

$$D_1={\left({\hat{\mathbf{z}}}_{k_0|k-1}-{\mathbf{z}}_i\right)}^T{S}_{k_0}^{-1}\left({\hat{\mathbf{z}}}_{k_0|k-1}-{\mathbf{z}}_i\right)<b,\forall i=1,\cdots ,m_k,$$

(15)

where $m_k$ is the total number of measurements. The set of validated measurements is defined as

$${\mathbf{Z}}_j=\left\{{\mathbf{z}}_1,{\mathbf{z}}_2,\cdots ,{\mathbf{z}}_{M_k}\right\},$$

where $M_k$ is the number of validated measurements in step k. The posterior probability distribution of each target obtained from the JPDA filter is a Gaussian mixture distribution. The state estimation is updated using a pseudo-innovation term

$$\tilde{\mathbf{z}}=\sum _{j=1}^{M_k}{\beta }_j({\mathbf{z}}_j-{\hat{\mathbf{z}}}_{k_0|k-1}),$$

(16)

e.g., a weighted sum of the original innovation terms, where $\beta$ represents the marginal association probability from measurement j and ${\hat{z}}_{k,j}$ is the predicted measurement track. Next, the pseudo-innovation term is applied to the standard EKF in the measurement update stage as follows:

$$\begin{array}{c}{\hat{\mathbf{x}}}_{k_0|k_0}={\hat{\mathbf{x}}}_{k_0|k-1}+K_k\sum _{j=1}^{M_k}{\beta }_j({\mathbf{z}}_j-{\hat{\mathbf{z}}}_{k_0|k-1})\hfill \\ P_{k_0|k_0}=P_{k_0|k-1}-K_k(1-{\beta }_0)S_{k_0}{K}_k^T+K_k{\overline{P}}_k{K}_k^T\hfill \end{array}$$

(17)

where $K_k=P_{k_0|k-1}{H}^T{S}_{k_0}^{-1}$ and

$$\begin{array}{c}\hfill {\overline{P}}_k=\sum _{j=1}^{M_k}\left[{\beta }_j({\mathbf{z}}_j-{\hat{\mathbf{z}}}_{k_0|k-1}){({\mathbf{z}}_j-{\hat{\mathbf{z}}}_{k_0|k-1})}^T-\overline{\mathbf{z}}{\overline{\mathbf{z}}}^T\right].\end{array}$$

(18)

For track maintenance, history-based logic is used; in other words, a tentative track is confirmed to be a track if it is associated with at least one measurement N times out of M consecutive steps, while a confirmed track is eliminated if association fails n times out of m steps.

3.2.2. Track Update with Time Alignment

Considering the time discrepancy between the evaluation for estimation $t_k$ and the measurement timestamped $c_i\left(t_m\right)$, the measurement can be regarded as a delayed detection. In the second period [$k_0,k]$, a track update is performed using state prediction to time step k. For track fusion, all tracks must be evaluated at the same time in order to perform T2TF. Because each local tracker updates its measurement at $k_{i,0}$, the posteriori should be further propagated such that all local tracks have the same time alignment. The main objective of the track update is to obtain the state prediction and covariance values, respectively ${\hat{\mathbf{x}}}_k$ and $P_k$:

$$\begin{array}{cc}\hfill {\hat{\mathbf{x}}}_k:={\hat{\mathbf{x}}}_{k|k_0}& =\mathbf{f}\left({\hat{\mathbf{x}}}_{k_0|k_0}\right),\hfill \\ \hfill P_k:=P_{k|k_0}& =F_{k,k_0}{P}_{k_0|k_0}{F}_{k,k_0}^T+Q_{k,k_0}.\hfill \end{array}$$

(19)

Notably, the large amount of clock offset due to synchronization failure may involve a measurement time step that is far from the current track interval. If the negative time offset involves $k_0<k-l$ for $l\ge 1$, then the measurements can be regarded as out-of-sequence measurements (OOSMs) such that the measurements obtained in the previous interval $k-1<k_0<k-l+1$ correspond to the current time step k. In this case, the OOSMs can be handled based on the approach presented in [45,46]. First, we consider the state estimates at k. The track automatically traverses the states and covariance from k − 1 to k ${\mathbf{x}}_{k|k}={\mathbf{x}}_{k|k-1}=\mathbf{f}\left({\mathbf{x}}_{k-1}\right)$ and $P_{k|k}=P_{k|k-1}=F_{k,k-1}{P}_{k-1}{F}_{k,k-1}^T+Q_{k,k-1}$, as no measurement exists in the current update interval $[k-1,k]$. In the retrodiction step, the current state and covariance are propagated back from k to $k_0$:

$${\hat{\mathbf{x}}}_{k_0|k}=F_{k_0,k}{\hat{\mathbf{x}}}_{k|k}$$

(20)

$$P_{k_0|k}=F_{k_0,k}\left[P_{k|k}+Q_{k,\tau }-P_{k_0|k}^{xv}-{\left(P_{k_0|k}^{xv}\right)}^T\right]F_{k_0,k}^T$$

(21)

where $F_{k_0,k}=F(t_{k_0},t_k)$ represents the backward state transition matrix from k to $t_{k_0}$, $Q(k,k_0)$ is the covariance matrix for the process noise, and $P_{k_0|k}^{xv}$ is defined as

$$\begin{array}{cc}\hfill & \hfill P_{k_0|k}^{xv}=Q_{k,k_0}-P_{k|k-l}{\left(S_k^{*}\right)}^{-1}{Q}_{k,k_0}\hfill \\ \hfill & \hfill {\left(S_k^{*}\right)}^{-1}=P_{k|k-l}^{-1}-P_{k|k-l}^{-1}{P}_{k|k}{P}_{k|k-l}^{-1}.\hfill \end{array}$$

(22)

In the second step, retro-correction is performed to correct the current state and state covariance using the OOSMs with a JPDA algorithm. The Kalman gain and innovation matrix at time step $k_0$ can be rewritten as

$$\begin{array}{c}K(k,k_0)=P_{k_0|k}^{xz}{S}_{k_0}^{-1}\hfill \\ S_{k_0}=H{P}_{k_0|k}{H}^T+R,P^{xz}=[P_{k|k}-P_{k_0|k}^{xv}]F_{k_0,k}^T{H}^T,\hfill \end{array}$$

(23)

where $H\left(\tau \right)=\frac{\partial h}{\partial x}$ is the observation Jacobian matrix and $R\left(\tau \right)$ is the covariance matrix for the OOSMs. Now, the corrected state can be derived as follows:

$${\hat{\mathbf{x}}}_{k|k_0}={\hat{\mathbf{x}}}_{k|k}+K(k,k_0)\overline{z}$$

(24)

where $\overline{z}={\sum }_{j=1}^{M_k}{\beta }_j(z_j-h\left(x_{k_0|k}\right))$ and

$$\begin{array}{cc}\hfill P_{k|k_0}=& P_{k_0|k}-(1-{\beta }_0)K(k,k_0)SK{(k,k_0)}^T\hfill \\ & +K(k,k_0)\left[\sum _{j=1}^{M_k}{\beta }_j({\mathbf{z}}_j-h\left({\hat{\mathbf{x}}}_{k_0|k}\right)){({\mathbf{z}}_j-h\left({\hat{\mathbf{x}}}_{k_0|k}\right))}^T-\overline{\mathbf{z}}{\overline{\mathbf{z}}}^T\right]K{(k,k_0)}^T.\hfill \end{array}$$

(25)

In contrast, a large positive time offset may involve $k_0>k+l-1$. Because the tracker can only process measurement times smaller than the current time, the invalid measurement is intentionally delayed up to one step to ensure that it can be suitably processed.

3.3. Track Fusion

When the FC receives local tracks, the tracks are fused through track fusion to produce global tracks. Because local tracks have different state-space models with different dimensions, i.e., ${\mathbf{x}}_a=[x,\dot{x},y,\dot{y},z,\dot{z}]$ and ${\mathbf{x}}_p=[x,\dot{x},y,\dot{y}]$, the association must be performed only on their shared state space. This study adopts a practical T2TF technique [37]. Track association first compares local tracks through the statistical association test. Considering the mapping $x_p=G{x}_a$, the state vector of the active sensor track can be partitioned as ${\mathbf{x}}_a={\mathbf{x}}_{a,1}\oplus {\mathbf{x}}_{a,2}$, where ${\mathbf{x}}_{a,1}=[x,\dot{x},y,\dot{y}]\in {\mathbb{R}}^4$ and ${\mathbf{x}}_{a,2}=[z,\dot{z}]\in {\mathbb{R}}^2$ are partitioned state vectors. The corresponding covariance matrix $P_{a,1}\in {\mathbb{R}}^{4\times 4}$ and $P_{a,2}\in {\mathbb{R}}^{2\times 2}$ can be expressed as

$$P_a=\left[\begin{array}{cc}{P}_{a,1}& P_{a,12}\\ P_{a,21}& P_{a,2}\end{array}.\right]$$

(26)

Based on the partition, the association test validates the statistical closeness between two tracks through comparison with the gating threshold:

$$D={\left({\hat{\mathbf{x}}}_{a,1}-{\hat{\mathbf{x}}}_b\right)}^T{P}_{a,1}^{-1}\left({\hat{\mathbf{x}}}_{a,1}-{\hat{\mathbf{x}}}_b\right)<b,$$

(27)

where b is the gating threshold. If the hypothesis is validated, the two tracks can be fused using the intersection method [47]:

$${\hat{\mathbf{x}}}_{F,s}=P_F(\omega {\left(P_{a,1}\right)}^{-1}{\hat{\mathbf{x}}}_{a,1}+(1-\omega )P_b^{-1}{\hat{\mathbf{x}}}_b),$$

(28)

$$P_{F,1}^{-1}=\omega {\left(P_{a,1}\right)}^{-1}+(1-\omega )P_b^{-1},$$

(29)

where $\omega$ represents a weight parameter chosen from within $(0,1)$. After fusion in the shared state space, the complement state is added from the active sensor track ${\hat{\mathbf{x}}}_{F,disjoint}={\hat{\mathbf{x}}}_{a,2}$ as

$${\hat{\mathbf{x}}}_F={\hat{\mathbf{x}}}_{F,s}\oplus {\hat{\mathbf{x}}}_{F,disjoint},P_F=\mathrm{diag}\{P_{F,1},P_{a,2}\}.$$

(30)

4. Impact Analysis

4.1. Influence on Local Tracking and Association

The local tracker accommodates inaccurate measurement time information and may corrupt estimated tracks, thereby adversely influencing the estimation accuracy and association consistency. This section discusses the influence of the time error on the local tracking process. Here, $k_0^{*}$ is the ideal time index at which the measurement is taken, while ${(·)}^{*}$ represents the variables for the ideal case with $\Delta =0$.

4.1.1. Influence on Measurement-to-Track Association and Filtering

In phase $[k-1,k_0]$, the time error may influence the association and filtering processes. For association, the predicted measurement track ${\hat{\mathbf{z}}}_{k_0|k-1}$ and covariance matrix should be propagated from the previous time step $k-1$ to $k_0$. Supposing that the local track in the previous step is accurate and that the relation

$${\hat{\mathbf{x}}}_{k-1}^{*}\approx {\hat{\mathbf{x}}}_{k-1},P_{k-1}^{*}\approx P_{k-1}$$

(31)

is satisfied, then, considering the time error $\Delta <0$, the update interval for measurement prediction satisfies $t_{k_0^{*}}-t_{k-1}>t_{k_0}-t_{k-1}$. The deviation between the predicted measurements and covariance for ideal and corrupted tracks are

$$\begin{array}{cc}\hfill \left|\right|{\hat{\mathbf{z}}}_{k_0^{*}|k-1}-{\hat{\mathbf{z}}}_{k_0|k-1}\left|\right|& =\left|\right|H({\hat{\mathbf{x}}}_{k_0^{*}|k-1}-{\hat{\mathbf{x}}}_{k_0|k-1})\left|\right|=\left|\right|H({\mathbf{f}}_{k_0^{*}|k-1}\left({\hat{\mathbf{x}}}_{k-1}\right)-{\mathbf{f}}_{k_0|k-1}\left({\hat{\mathbf{x}}}_{k-1}\right))\left|\right|\hfill \\ \hfill & \approx \left|\right|H(F_{k_0^{*}|k-1}-F_{k_0|k-1}){\hat{\mathbf{x}}}_{k-1}\left|\right|=\left|\right|H\Delta F_{k_0^{*}|k_0}{\hat{\mathbf{x}}}_{k-1}\left|\right|,\hfill \end{array}$$

(32)

where $\left|\right|·\left|\right|$ denotes the 2-norm and $\Delta F_{k_0^{*}|k_0}$ is the state transition error matrix. Equation (32) indicates that the predicted measurement error from the ideal case increases with respect to $\Delta$ as it is amplified by $\Delta F_{k_0^{*}|k_0}$. In turn, the incorrect measurement prediction affects the estimation accuracy. Furthermore, the covariance of innovation matrix can be defined as

$$|S_{k_0^{*}}|=|H_k{P}_{k_0^{*}|k-1}{H}^T+R=H_k{P}_{k_0^{*}|k_0}{P}_{k_0|k-1}{H}^T+R|>|S_{k_0}|.$$

(33)

Equation (33) reflects that the update to the innovation is minor in the corrupted track case. The small innovation update affects the association test, i.e.,

$$|S_{k_0^{*}}|>|S_{k_0}|\Rightarrow |S_{k_0^{*}}^{-1}|<|S_{k_0}^{-1}|,$$

(34)

which may imply that the statistical distance with the same deviation between the actual measurement and prediction becomes significant with $|S_{k_0}^{-1}|$. This may lead to unsuccessful validation of the target measurement and deteriorate the consistency of the track association. In addition to the propagation error associated with kinematic mismatch and process error, the prediction error worsens as well.

4.1.2. Influence on Track Update with Time Alignment

In the track update phase, the updated track may be corrupted by the inaccurate update interval between $k_0$ and k. For simplicity, suppose that the synchronization error $\Delta =c_i\left(t_m\right)-t_m=t_{k_0}-t_{k_0^{*}}<0$ is negative and that the target measurement is validated as ${\beta }_0=0$. The measurement update step corrects the state estimates and covariance matrix using latest measurement at $k_0$. Given this information, we assume that the a posteriori estimates and covariance matrix are similar to those of the ideal case, as

$${\hat{\mathbf{x}}}_{k_0^{*}|k_0^{*}}^{*}\approx {\hat{\mathbf{x}}}_{k_0|k_0},P_{k_0^{*}|k_0^{*}}\approx P_{k_0|k_0}.$$

(35)

Now, the track update propagates the posteriori $\hat{\mathbf{x}}\left(k_0\right|k-1)$ to the current time k. According to Equation (19), the estimation error from ideal estimates satisfies

$$\left|\right|{\hat{\mathbf{x}}}_{k|k_0^{*}}^{*}-{\hat{\mathbf{x}}}_{k|k_0}\left|\right|=\left|\right|f\left({\hat{\mathbf{x}}}_{k_0^{*}|k_0^{*}}^{*}\right)-f\left({\hat{\mathbf{x}}}_{k_0|k_0}\right)\left|\right|\approx \left|\right|F_{k,k_0^{*}}{\hat{\mathbf{x}}}_{k_0^{*}|k_0^{*}}^{*}-F_{k,k_0}{\hat{\mathbf{x}}}_{k_0|k_0}^{*}\left|\right|=\left|\right|\Delta F_{k_0^{*},k_0}{\hat{\mathbf{x}}}_{k_0,k_0}\left|\right|.$$

(36)

Equation (36) indicates that the deviation from the ideal track increases by $\Delta F_{k_0^{*},k_0}$. In particular, $\Delta F_{k_0^{*},k_0}=F_{-\Delta }$ propagates the track further along the target velocity direction when $\Delta <0$. Furthermore, the covariance matrix can be written as

$$\begin{array}{cc}\hfill |P_{k|k_0}|& =|F_{k,k_0}{P}_{k_0|k_0}{F}_{k,k_0}^T+Q_{k,k_0}|=|\left(F_{k,k_0^{*}}{F}_{k_0^{*},k_0}\right)P_{k_0^{*}|k_0^{*}}{\left(F_{k,k_0^{*}}{F}_{k_0^{*},k_0}\right)}^T+Q_{k,k_0^{*}}{Q}_{k_0^{*},k_0}|\hfill \\ \hfill & =|F_{k_0^{*},k_0}\left(F_{k,k_0^{*}}{P}_{k_0^{*}|k_0^{*}}{F}_{k,k_0^{*}}^T\right){\left(F_{k_0^{*},k_0}\right)}^T+Q_{k,k_0^{*}}{Q}_{k_0^{*},k_0}|\ge |P_{k|k_0^{*}}|,\hfill \end{array}$$

(37)

which means that the covariance grows as the time error $\Delta$ acts on the local measurement. In turn, the prediction step for time alignment leads to deviation of the state estimates from the actual target, and this trend becomes more pronounced as the time offset increases.

4.2. Impact on Track-to-Track Association

As described in Section 4.1, the time synchronization error increases the tracking error and error covariance in local tracks. A larger time error leads to increased error covariance from the local filter, and the fused covariance is affected as well; therefore, the time error results in tracking degradation. In addition, the degraded quality of local tracks can affect the track-to-track association. The increase in time error reflects the deviation between the local track estimates and the reduced association between local tracks. Suppose that two local sensors detect the target at $t_{k_0^{*}}$ with different timestampings by local clocks; let $k_0$ and $k_1$ denote the measurement time steps corrupted by inaccurate local clocks. Then, Equation (36) yields

$$\begin{array}{cc}\hfill & \hfill {\hat{\mathbf{x}}}_{k|k_0}=F_{k,k_0}{\hat{\mathbf{x}}}_{k_0^{*}|k_0^{*}},\hfill \\ \hfill & \hfill {\hat{\mathbf{x}}}_{k|k_1}=F_{k,k_1}{\hat{\mathbf{x}}}_{k_0^{*}|k_0^{*}}.\hfill \end{array}$$

(38)

Considering the maximum time deviation between local clocks (9), the deviation between local tracks can be expressed as

$$|{\hat{\mathbf{x}}}_{k|k_0}-{\hat{\mathbf{x}}}_{k|k_1}|=|F_{k_1,k_0}{\hat{\mathbf{x}}}_{k_0^{*}|k_0^{*}}|\le |F_{-2\Delta }{\hat{\mathbf{x}}}_{k_0^{*}|k_0^{*}}|.$$

(39)

Equation (39) implies that the deviation increases as $F_{-2\Delta }$ amplifies the upper bounds between local tracks.

5. Simulation Study

Numerical simulations were performed to validate the impact analysis. We considered the scenario of an intruder drone being detected near an airport, as shown in Figure 3. The target is moving from ${\mathbf{p}}_0=[600,600,10]$ with constant velocity ${\mathbf{v}}_t=[10\sqrt{3},10,0.2]$. The radar is located at the center of the surveillance space, and is capable of 360-degree field of view functionality through the use of a rotating receiver antenna, while the RF sensors are distributed to continuously search the space. The sensor detects the target in the cluttered environment, and local trackers process the cluttered measurements and generate local tracks. The RF tracker receives measurements from the RF sensor once in two cycles. The tracker coasts the track alternately when the measurement is not transmitted. MATLAB Sensor Fusion Toolbox was used to model the sensors [48], and the sensor specifications and system parameters are summarized in

Table 2. Tracking algorithm parameters.

	Radar Tracker	RF Tracker
Update rate	1 Hz	1 Hz
Filter	3D-CV-EKF [41]	2D-CV-EKF [41]
Association threshold, b	30	30
M/N logic parameters for track maintenance	$M=3,N=5$	$M=3,N=5$
M/N logic parameters for track deletion	$M=3,N=3$	$M=3,N=3$

.

For clock offsets, we consider the worst-case scenario, where the radar clock has negative time offset and the RF sensor has positive time offset. The local clocks are considered to be synchronized when the clock offsets are bounded below the criterion under the required accuracy $|{\Delta }_t|=|c_i\left(t\right)-t|<1$ ms, which is obtained from the statistical model (Equation (4)). Otherwise, the networks are unsynchronized. The time offsets are increased to monitor the qualitative trends of the performance with respect to the time error.

The performance is evaluated using three metrics. The average RMSE refers to the localization error of the track result compared with the ground truth. Lower RMSE scores correspond to higher tracking precision. In this study, the RMSE criterion is set as 2 m.

$$AvgRMSE=\frac{1}{T}\sum _t^{T_f}\sqrt{{(x\left(t\right)-{\hat{x}}_{track}\left(t\right))}^2+{(y\left(t\right)-{\hat{y}}_{track}\left(t\right))}^2+{(z\left(t\right)-{\hat{z}}_{track}\left(t\right))}^2}$$

(40)

The GOSPA [25] is used to assess the tracking performance by simultaneously considering track completeness in terms of the localization error, miss target, and false tracks.

$$GOSPA={\left(lo{c}^p+mis{s}^p+flas{e}^p\right)}^{1/p}$$

(41)

Lastly, we consider the single integrated air picture (SIAP) [26], which helps to separately examine the tracking quality in terms of consistency and correctness. This metric includes two metrics; the Completeness indicates the percentage of number of tracked targets to the number of actual targets related to missed targets, while the Spuriousness is the percentage of the number of false tracks out of the number of total tracks.

$$C_t\left(t\right)=\frac{n\left(truthtracked\right)}{n\left(trut{h}_t\right)},S_t=\frac{n\left(tracks\right)-n\left(assignedtracks\right)}{n\left(tracks\right)}$$

(42)

5.1. Comparison between Synchronized and Unsynchronized Sensor Networks

Figure 4 shows the target tracking results when the sensors are time-synchronized under the required precision $|\Delta |<1$ ms. The track trajectory, presented in an identical color in Figure 4a, indicates that the fused track is consistently derived during the scenario. As shown in Figure 4b, the confirmation times in local and fused tracks are different. The radar track is confirmed at the third time step, while the RF tracker confirms its local track at the fifth time step when the measurement arrives three times. Finally, the fused track is derived from the fifth time step when two local tracks are confirmed. Local tracks exhibit large tracking errors and missed targets in a few initial steps, while the fused track exhibits improved tracking accuracy.

Table 3. Comparison results in performance metrics.

	Ideal, ${\Delta }_{\mathit{t}}=0$			Synchronized, $|{\Delta }_{\mathit{t}}|=1$ ms			Unsynchronized ($|{\Delta }_{\mathit{t}}|=0.5$ s)
	Radar	RF	Fused	Radar	RF	Fused	Radar	RF	Fused
RMSE	1.9819	1.549	0.7815	1.982	1.892	1.186	7.662	7.279	7.0064
GOSPA	10.811	9.276	6.857	11.278	10.468	8.371	29.362	29.195	30.230
$C_t$ (%)	100	100	100	92.86	96.15	100	25.00	30.77	15.38
$S_t$ (%)	0	0	0	7.14	3.85	0	75.00	69.23	84.62

summarizes the performance measures. Compared with the ideal case without any time error, the overall target tracking performance is maintained within satisfactory limits.

In contrast, the target tracking system exhibits reduced performance in the unsynchronized sensor network. Figure 5 shows that each local tracker generates a local track but displays large tracking errors when a significant time error ($|\Delta |=0.5$ s) is imposed on the local clocks. Time alignment in track update causes the track to deviate from the measurements, introducing a large RMSE. Consequently, the completeness metric reflects that the track is not correctly assigned to the target and labels it as a false track. Furthermore, the track fusion is significantly impaired. The RMSE significantly increases and exceeds the tracking requirement. In addition, the consistency of the fused track deteriorates in most of the time intervals. Figure 5a illustrates that the fused track trajectory is not consistent. From the fifth to tenth time steps, the fusion center cannot fuse the two local tracks and yields two separate tracks, potentially because of the enlarged error bound in Equation (39) as the time error increases. This result indicates that the tracking performance is significantly affected by the time error when the local clocks are no longer synchronized.

5.2. Performance under Increasing Synchronization Errors

To examine the influence of the synchronization error on the target tracking system, we implemented the same tracking scenario with synchronization errors increasing from the zeroth time step to the first. Figure 6 plots the average values of the performance measures for each episode. The local and fused tracks exhibit less sensitivity to the synchronization error when the offset is within the allowable offset $|\Delta |<1$ ms. However, the tracking degradation becomes noticeable when the synchronization error exceeds 100 ms. Considering the clock drift during the longer holdover interval, the holdover time at which significant performance degradation is observed can be estimated. For example, local clocks with an accuracy of 2 ppm (parts per million) take $0.1/(2\times {10}^{-6}/3600)\approx 14$ h to reach the time error of 100 ms. This time is reduced for low-cost clocks due to the large frequency stability in their oscillators.

Furthermore, the fused performance worsens compared with that of local tracks in the presence of significant time errors, which implies that fusion with tracks corrupted by synchronized time errors may result in reduced tracking quality. Notably, a correlation exists between the RMSE/GOSPA and SIAP metrics; RMSE and GOSPA grow significantly after 100 ms, and $C_t$ and $S_t$ indicate that the tracking system degrades the track consistency in this period. GOSPA incorporates the missed (completeness) and false (spuriousness) components. A large RMSE indicates that the distanced track becomes less coherent from a kinematic perspective, and the corresponding association with the central track may fail, as observed in Figure 5. The exponential increase in RMSE with the log-scale temporal error indicates that the estimation errors in the local and fusion tracks tend to be linearly proportional to the time error. This result is consistent with the impact analysis. Specifically, the approximate tendency derived in Equations (32), (36) and (39) shows the linear correlation between the performance degradation and time error.

5.3. Performance When the Target Is Maneuvering

To inspect the system performance when the target is maneuvering in the presence of synchronization error, a target tracking simulation was performed by varying the target’s motion. In this scenario, the target is in constant turning motion with identical speed $v_t=10$ m/s.

Figure 7 illustrates that the negative effects of time synchronization errors on the sensor fusion performance are aggravated when model mismatch occurs due to target maneuvering. In both cases, the tracking performance degrades as the time error increases. However, the degradation is faster in the maneuvering case, as the maximum synchronization error for meeting the performance requirement is reduced. This degradation may be attributable to a mismatch between the target motion in the tracking algorithm. The local tracker estimates the target based on the constant-velocity model in the EKF phase. The model error can be propagated along with the synchronization error in track estimates, thereby influencing the association and track updates, which may worsen when a large synchronization error is involved.

5.4. Performance under Speed Variations

As shown in Figure 8a, RMSE and GOSPA increase rapidly as the target speed increases. For example, the time to reach the RMSE error of 2 m is 0.24 s when the target speed is 10 m/s, while it is significantly reduced to 0.05 s when the target speed increases to 40 m/s. This phenomenon likely occurs because the velocity components in the state vector lead to sizable spatial transitions over time. The high speed of the target potentially introduces a significant position error in Equation (36). In addition, the track consistency degrades faster as the speed increases. Moreover, when the target speed increases, the robustness of sensor fusion against the time synchronization error is reduced as the completeness metric decreases. Figure 8c shows the empirical results of the allowable synchronization errors for track consistency and estimation accuracy. The blue shaded area represents the allowable time error for guaranteed estimation accuracy, while the grey shaded area denotes the allowable region of the consistent track. Both regions narrow as the target speed increases. The estimation accuracy region is smaller than the consistent track region, indicating that the tracking requirement is considerably tighter in this scenario. The consistency region can be widened as the association threshold is adaptively set or increased.

6. Conclusions

This study aimed to examine the influence of time synchronization errors on the performance of a target tracking system. The time synchronization error was mathematically modeled and imposed on local clocks as the clock offset. The target tracking system consisted of heterogeneous sensors under a T2TF architecture. The local tracking process included a track update step for time alignment to account for the asynchronous timing between measurements and tracks and to accommodate synchronization errors in generating local tracks. The impact of the time synchronization error on association, filtering, and track fusion was considered, and empirical results were obtained through simulations. Under linear target dynamics, the synchronization error linearly degraded the tracking error, resulting in inconsistent association. Variations in other factors, including target speed and maneuvering, also led to deterioration in tracking performance owing to time synchronization errors. The simulation results demonstrated that these negative effects on the target tracking system become pronounced when the error exceeds 100 ms. These results highlight the need for a resilient synchronization system that can ensure time synchronization across the network in order to maintain synchronization errors within an acceptable accuracy level. Joint estimation of time synchronization and target position in target tracking and sensor fusion processes can be implemented to mitigate the errors when a synchronization failure occurs. The findings of this work can be extended to other sensor network systems to clarify the influence of synchronization errors and facilitate system improvements. Further research could explore the development of state estimation techniques aimed at improving tracking performance in the presence of measurement delays, with a specific focus on addressing the impact of time synchronization in sensor networks.