An Overview of a Class of Clock Synchronization Algorithms for Wireless Sensor Networks: A Statistical Signal Processing Perspective

Abstract

Recently, wireless sensor networks (WSNs) have drawn great interest due to their outstanding monitoring and management potential in medical, environmental and industrial applications. Most of the applications that employ WSNs demand all of the sensor nodes to run on a common time scale, a requirement that highlights the importance of clock synchronization. The clock synchronization problem in WSNs is inherently related to parameter estimation. The accuracy of clock synchronization algorithms depends essentially on the statistical properties of the parameter estimation algorithms. Recently, studies dedicated to the estimation of synchronization parameters, such as clock offset and skew, have begun to emerge in the literature. The aim of this article is to provide an overview of the state-of-the-art clock synchronization algorithms for WSNs from a statistical signal processing point of view. This article focuses on describing the key features of the class of clock synchronization algorithms that exploit the traditional two-way message (signal) exchange mechanism. Upon introducing the two-way message exchange mechanism, the main clock offset estimation algorithms for pairwise synchronization of sensor nodes are first reviewed, and their performance is compared. The class of fully-distributed clock offset estimation algorithms for network-wide synchronization is then surveyed. The paper concludes with a list of open research problems pertaining to clock synchronization of WSNs.

1. Introduction

A wireless sensor network (WSN) is a group of spatially-distributed autonomous sensors, which monitor physical or environmental conditions, such as temperature, humidity, speed, pressure, etc., and then transmit the recorded data to a central computing unit for analysis. WSNs originate from military-oriented applications, such as battlefield surveillance and movement monitoring. Nowadays, WSNs have witnessed a rapid growth and have been implemented in a variety of promising applications, which can be classified as follows:

• Military [1]: battlefield surveillance; forces monitoring; battle damage assessment.
• Health [2,3]: tracking and monitoring patients; telemonitoring of human physiological data.
• Environment [4,5]: air pollution monitoring; water quality monitoring; forest fire detection.
• Home [6,7]: home automation; home appliances management and monitoring.
• Industry [8]: machine health monitoring; waste monitoring; data logging.

Most of these applications require the clocks of network nodes to be synchronized, because performing a joint task requires all of the nodes to operate on a common time scale. For instance, data logging is a basic operation that collects, processes and transmits data and is used for temporal and spatial monitoring of a habitat. The advantage of data logging in WSNs over the conventional data logging lies in the property of real-time data being collected, which requires some or all nodes in the network to share a common time frame.

Each sensor in a WSN operates independently and has its own clock. Even if the clocks of sensors are initially tuned perfectly, due to the imperfections of the clock oscillator, they may drift away from the ideal time as time evolves [9]. Hence, developing efficient clock synchronization protocols is critical in WSNs. In the literature, many clock synchronization algorithms rely on the clock information from the Global Positioning System (GPS). However, GPS is not ubiquitously available and requires a high-power receiver [10]. This fact makes it impractical to implement GPS technology in the clock synchronization of WSNs, since WSNs generally consist of small sensors that have limited energy. Furthermore, sensors sometimes are positioned in an environment where the GPS signal is not available, as is the case with underwater or underground sensors for water quality monitoring. For clock synchronization applications in computer networks, the network time protocol (NTP) represents one of the oldest and standard Internet timing protocols in current use [11]. However, NTP is still not suitable for WSNs, due to the lack of energy availability and unstable working environment in WSNs. Therefore, a variety of synchronization protocols have been particularly designed for WSNs. In general, these protocols perform clock synchronization via timing message exchanges among sensors. However, due to the power requirement for each individual sensor, sensors have a limited communication range, and they are not able to communicate with every other sensor. An example of a WSN with nine sensors is shown in Figure 1a, where each circle denotes a sensor and each edge represents the communication link (if present) between the corresponding pair of sensors.

Figure 1. Network topology for different synchronization approaches.

Figure 1. Network topology for different synchronization approaches.

Traditional synchronization protocols for WSNs generally include two steps: first, the synchronization is conducted within a pair of neighboring nodes; second, a multi-hop structure is built, such that the synchronization can be extended into all of the nodes through a layer-by-layer synchronization. One natural approach to deal with the multi-hop structure of WSNs is to construct a tree-based network. Specifically, as illustrated in Figure 1b, a sensor is selected as the reference node, then a spanning tree rooted at this node is built, and each sensor synchronizes with its parent pairwisely along the unique path from that sensor to the root. On the other hand, the cluster-based structure divides the network into several interconnected single-hop clusters, as graphically shown in Figure 1c. In each cluster, a reference node is selected, and all of the other nodes are pairwisely synchronized by the reference node. The reference nodes of different clusters act as gateways to adjust the local clock of different clusters into one common time frame. In general, the core of these protocols is based on timing message exchanges between a pair of nodes, in which accurate and efficient pairwise clock synchronization algorithms play a key role. In the literature, some widely-used clock synchronization protocols, such as the timing-sync protocol for sensor networks (TPSN) [12], the tiny/mini synchronization [13], the light-weight time synchronization (LTS) [14] and the flooding time synchronization protocol (FTSP) [15], employ a two-way message exchange mechanism to adjust the clock between any two nodes.

However, the traditional synchronization protocols suffer from large overhead in building and maintaining the spanning-tree or cluster structure. In addition, some nodes act as a root (in the spanning-tree structure) or as a gateway (in the cluster structure), and the failure of such nodes may lead to the failure of a large number of nodes connecting to it. To tackle this problem, several algorithms based on a fully-distributed communication topology have been proposed. In such algorithms, there are no special nodes, such as roots and gateways, and thus, no structure needs to be built. With all of the nodes performing exactly the same algorithm and communicating with their neighboring nodes, these protocols are robust to the topology changes and failures of nodes.

In this paper, the latest algorithms in the realm of clock synchronization for WSNs are surveyed following a statistical signal processing viewpoint. Specifically, the rest of the paper is organized as follows. Section 2 introduces the clock model and the two-way message exchange mechanism for pairwise clock synchronization. Based on the two-way message exchange mechanism, the state-of-the-art pairwise clock synchronization algorithms under Gaussian random delays are investigated in Section 3. Section 4 overviews the pairwise clock synchronization algorithms in the presence of exponential random delays. Section 5 surveys the signal processing techniques employed in WSNs when the random delays are unknown. In Section 6, synchronization algorithms that assume a fully-distributed operation are discussed. Finally, Section 7 concludes the paper and provides some open research topics.

In the literature, surveys for clock synchronization in WSNs have been presented by [9,10,16,17]. The focus of [10,16] lies in the clock synchronization protocols that are applied in WSNs. The survey paper [17] provides a technical overview of the history, recent advances and challenges in distributed clock synchronization for distributed wireless networks, while this article presents synchronization algorithms with emphasis on the mathematical and statistical techniques employed for clock synchronization parameters estimation. The work in [9] surveys the latest advances in the clock synchronization of WSNs following a signal processing viewpoint. However, only pairwise synchronization methods are investigated in [9], while our paper discusses both pairwise and fully-distributed synchronization algorithms.

2. System Model for Pairwise Clock Synchronization

In this section, the clock model for each senor node is first introduced and the notions of clock offset and skew are described. Then, the classic mechanism for message exchange between two adjacent nodes, namely the two-way message exchange mechanism, is presented.

2.1. Clock Offset and Skew

As discussed earlier, each sensor node in a WSN works independently and has its own clock. Ideally, the clock of a sensor node is modeled as $c\left(t\right)=t$, where t represents the reference time [9]. However, due to the imperfections of the clock oscillator, as well as the environmental factors, such as pressure, temperature and hardware aging, the clock is subject to change as time evolves. Generally, the clock function of node i can be expressed as:

$$c_i\left(t\right)={\theta }_i+f_i·t$$

(1)

where ${\theta }_i$ and $f_i$ denote the clock offset (phase difference) and the clock skew (frequency difference) for sensor node i, respectively [9]. In this way, if the clock of Node A is selected to be the reference time, it follows that:

$$c_B\left(t\right)=\theta +f·c_A\left(t\right)$$

where θ and f stand for the clock offset and skew between Nodes A and B, respectively. A graphical interpretation of the relationship between the clocks of Nodes A and B is shown in Figure 2. It can be seen that if $\theta =0$ and $f=1$, then Nodes A and B are perfectly synchronized. Otherwise, the synchronization process aims to estimate the clock offset θ and skew f, such that Node B can adjust its local clock to the reference time (Node A). As time evolves, the clock offset and skew are subject to change due to various reasons, such as the changes induced by the environment and sensor hardware aging. Therefore, the clock parameters are synchronized periodically, such that they are tuned up-to-date.

Figure 2. Clocks of sensor nodes.

Figure 2. Clocks of sensor nodes.

2.2. Two-Way Message Exchange Mechanism

The two-way message exchange is a classic mechanism for communicating the timing messages between two adjacent nodes and has been widely used in the literature [12,13,14,15]. For pairwise synchronization, one of the sensor nodes is selected as the reference node. For example, a two-way message exchange model between Node A and Node B is shown in Figure 3 in the situation when Node A is serving as the reference node. During message exchange round i, the synchronization begins at Node A, and a synchronization message containing the sending time $T_i^a$ is sent to Node B. Next, Node B records the reception of the message at its clock time $T_i^b$ and sends an acknowledgment message to Node A at $T_i^c$. The acknowledgment message contains the timestamps $T_i^b$ and $T_i^c$. At last, Node A receives the acknowledgment message at $T_i^d$, and the message exchange round i ends. This process is repeated N times, where N stands for the required number of observations.

Based on the procedure depicted above, the clock synchronization problem can be mathematically modeled as

$$\begin{array}{cc}\hfill T_i^b=& f(T_i^a+d+X_i)+\theta \hfill \\ \hfill T_i^c=& f(T_i^d-d-Y_i)+\theta \hfill \end{array}$$

(2)

where θ and f represent the clock offset and clock skew, respectively. Moreover, $X_i$ and $Y_i$ denote the random delays from Node A to Node B and from Node B to Node A, respectively, and d stands for the fixed portion of delays. In addition, $T_i^a,T_i^d$ and $T_i^b,T_i^c$ are time stamps recorded at Nodes A and B, respectively.

Figure 3. Two-way message exchange mechanism (with skew).

Figure 3. Two-way message exchange mechanism (with skew).

If we consider only the clock (phase) offset between two adjacent nodes, i.e., $f=1$, the system model in Equation (2) can be expressed as:

$$\begin{array}{cc}\hfill T_i^b=& T_i^a+d+X_i+\theta \hfill \\ \hfill T_i^c=& T_i^d-d-Y_i+\theta \hfill \end{array}$$

(3)

The equations in Equation (3) can be further simplified as:

$$\begin{array}{cc}\hfill U_i=& d+\theta +X_i\hfill \\ \hfill V_i=& d-\theta +Y_i\hfill \end{array}$$

(4)

where $U_i$ and $V_i$ contain the clock information and are defined as $U_i=T_i^b-T_i^a$ and $V_i=T_i^d-T_i^c$, respectively.

3. Pairwise Clock Synchronization under Gaussian Delays

In general, probability density function (PDF) models have been employed for modeling the random portion of delays in WSNs. Examples of PDFs that have been adopted in the literature include exponential, Gaussian, Weibull and Gamma [18,19,20,21]. In this section, we use the Gaussian distribution to characterize the random delay in WSNs, i.e., $X_i$ and $Y_i$ are Gaussian random variables with parameters $({\mu }_1,{\sigma }_1)$ and $({\mu }_2,{\sigma }_2)$, respectively. The reasons for selecting the Gaussian distribution for modeling the random delays are as follows:

• The central limit theorem (CLT) states that the PDF of the sum of a large number of independent and identically-distributed (i.i.d.) random variables is approximately normally distributed. Therefore, the Gaussian model is appropriate if the random delays are assumed to be the summation of multiple independent random variables.
• Experimental results based on two Texas Instruments ez430-RF2500 evaluation boards were recorded in [22] to demonstrate the fitness of the Gaussian distribution in modeling the random portion of delays in WSNs.

The most representative pairwise clock synchronization algorithms in the literature focused on developing maximum likelihood estimators (MLEs) for the clock offset and skew. The MLE is one of the most widely-used estimators in the area of parameter estimation, since it achieves asymptotically the Cramér–Rao lower bound (CRLB).

Based on the two-way message exchange model in Equation (4), where only the clock offset is considered, [18] derived the MLE for clock offset by assuming a known fixed delay d and symmetric random delays, i.e., ${\mu }_1={\mu }_2=\mu$ and ${\sigma }_1={\sigma }_2=\sigma$. In this framework, the likelihood function is expressed as:

$$p(U_i,V_i|\theta )={\left(2\pi {\sigma }^2\right)}^{-N}{e}^{-\frac{1}{2{\sigma }^2}\left[{\displaystyle \mathrm{\sum }_{i=1}^N}{(U_i-d-\theta -\mu )}^2+{\displaystyle \mathrm{\sum }_{i=1}^N}{(V_i-d+\theta -\mu )}^2\right]}$$

The MLE of clock offset is derived by differentiating the log-likelihood function with respect to θ and setting the corresponding derivative to zero, and it assumes the expression:

$${\theta }_{\text{ML}}=\frac{\overline{U}-\overline{V}}{2}$$

(5)

where $\overline{U}$ and $\overline{V}$ stand for the sample means of observations ${\left\{U_i\right\}}_{i=1}^N$ and ${\left\{V_i\right\}}_{i=1}^N$, respectively. It can be verified that the estimator Equation (5) is unbiased and achieves the CRLB. The MLE estimator Equation (5) represents also the minimum variance unbiased estimator (MVUE).

Following the same assumptions, i.e., a known fixed delay and symmetric random delays, the joint maximum likelihood estimates of the clock offset and skew were also proposed in [18] based on the two-way message exchange model in Equation (2). Specifically, the likelihood function is first formulated as:

$$p(U_i,V_i|\theta ,f^{\prime })={\left(2\pi {\sigma }^2\right)}^{-N}{e}^{-\frac{1}{2{\sigma }^2}{\displaystyle \mathrm{\sum }_{i=1}^N}\{{[f^{\prime }(\theta -T_i^b)+(T_i^a+d)]}^2+{[f^{\prime }(\theta -T_i^c)+(T_i^d-d)]}^2\}}$$

where $f^{\prime }=1/(1+f)$. Differentiating the log-likelihood function with respect to θ leads to an equation of θ in terms of $f^{\prime }$. In addition, differentiating the log-likelihood function with respect to $f^{\prime }$ results in an equation of $f^{\prime }$ in terms of θ. The joint estimates of θ and $f^{\prime }$, denoted as ${\widehat{\theta }}_{\text{ML}}$ and ${\widehat{f}}_{\text{ML}}^{\prime }$, can be obtained by combining those two equations, and the MLE of f is obtained as ${\widehat{f}}_{\text{ML}}=1/(1+{\widehat{f}}_{\text{ML}}^{\prime })$, due to the invariance principle [23].

However, in some practical cases, the value of the fixed delay d is unknown and sometimes even represents a parameter that needs to be estimated, as is the case with the node localization application [24]. Therefore, efficient algorithms for joint estimation of the clock offset, clock skew and fixed delay are of great interest. Based on the two-way message exchange mechanism in Equation (2), Leng et al. [24] extended the result in [18] by assuming an unknown fixed delay d. First, the signaling model Equation (2) is re-formulated in matrix form as:

$$\underset{\stackrel{\mathrm{\Delta }}{=}{\mathbf{C}}_r}{\underbrace{\left[\begin{array}{c}{T}_1^a\\ ⋮\\ T_N^a\\ -T_1^d\\ ⋮\\ -T_N^d\end{array}\right]}}+d·\mathbf{1}=\underset{\stackrel{\mathrm{\Delta }}{=}{\mathbf{C}}_h}{\underbrace{\left[\begin{array}{cc}{T}_1^b& -1\\ ⋮& ⋮\\ T_N^b& -1\\ -T_1^c& 1\\ ⋮& ⋮\\ -T_N^c& 1\end{array}\right]}}\underset{\stackrel{\mathrm{\Delta }}{=}\mathbf{\Phi }}{\underbrace{\left[\begin{array}{c}{\varphi }_1\\ {\varphi }_0\end{array}\right]}}-\underset{\stackrel{\mathrm{\Delta }}{=}\mathbf{Z}}{\underbrace{\left[\begin{array}{c}{X}_1\\ ⋮\\ X_N\\ Y_1\\ ⋮\\ Y_N\end{array}\right]}}$$

where ${\varphi }_0\stackrel{\mathrm{\Delta }}{=}\theta /f$, ${\varphi }_1\stackrel{\mathrm{\Delta }}{=}1/f$ and $\mathbf{1}={[1,1,\cdots ,1]}^T$ with dimension $2N\times 1$.

Since $X_i$ and $Y_i$ are independent Gaussian random variables, i.e., $X_i\sim \mathcal{N}(0,{\sigma }^2),Y_i\sim \mathcal{N}(0,{\sigma }^2)$, the log-likelihood function for Φ and d is expressed as:

$$\text{ln} p({\mathbf{C}}_r,{\mathbf{C}}_h|\mathbf{\Phi },d)=\text{ln}\frac{N}{2\pi {\sigma }^2}-\frac{\left|\right|{\mathbf{C}}_r+d·\mathbf{1}-{\mathbf{C}}_h\mathbf{\Phi }{\left|\right|}^2}{2{\sigma }^2}$$

For a fixed d, the MLE of Φ is given by:

$$\widehat{\mathbf{\Phi }}\left(d\right)={\left({\mathbf{C}}_h^T{\mathbf{C}}_h\right)}^{-1}{\mathbf{C}}_h^T({\mathbf{C}}_r+d\mathbf{1})$$

(6)

Plugging Equation (6) into the log-likelihood function results in a compressed likelihood function with only one variable d. The MLE of d is derived by taking the derivative over the resulting log-likelihood function with respect to d, and the MLE of Φ is obtained by plugging the corresponding estimate of d back into Equation (6). Finally, the MLEs of θ and f are obtained from the MLE of Φ using the invariance principle [23].

4. Pairwise Clock Synchronization under Exponential Delays

In the literature, the exponential distribution is also commonly exploited to represent the random portion of delays in WSNs. The reasons behind choosing the exponential distribution include:

• For the point-to-point hypothetical reference connection (HRX) between two nodes, a single-server M/M/1queue can appropriately represent the aggregate link delay, where the random delays are modeled as independent exponential random variables [25].
• Experiment results were carried out in [26,27] to verify the appropriateness of choosing the exponential distribution to characterize the random delays in WSNs.
• Among all distributions with a fixed mean in the support $[0,+\mathrm{\infty })$, the exponential distribution achieves the maximum differential entropy, and thus, it is the least informative.

The most representative clock offset estimation methods for pairwise synchronization in the presence of exponential delays are the maximum likelihood estimator (MLE), the best linear unbiased estimator (BLUE) and the minimum variance unbiased estimator (MVUE), and they will be first reviewed in this section. The MLE for joint estimation of the clock offset and skew will be investigated later. Finally, an important statistical concept, referred to as the confidence interval for the clock offset, will be investigated.

4.1. Clock Offset Estimation under Exponential Delays

4.1.1. Maximum Likelihood Estimator

In the two-way message exchange mechanism Equation (4) where only clock offset exists, we assume that $X_i$ and $Y_i$ are exponentially-distributed random variables with means ${\lambda }_1$ and ${\lambda }_2$, respectively. Assuming a known fixed delay, Abdel-Ghaffar [25] reported that the MLE of the clock offset does not exist when the exponential random delays are assumed to be known and symmetric, i.e., ${\lambda }_1={\lambda }_2=\lambda$, since the likelihood function does not admit a unique maximum. However, Jeske [28] derived a closed-form solution for the MLE of clock offset by assuming an unknown fixed delay and an unknown mean for the exponential delay. In this way, a joint estimation of $\theta ,d$ and λ can ensure a unique maximum of the likelihood function. Specifically, assuming symmetric exponential delays with a common mean λ, the likelihood function for ${[d,\theta ,\lambda ]}^T$ is expressed as:

$$p(U_i,V_i|d,\theta ,\lambda )={\lambda }^{-2N}{e}^{-\left({\displaystyle \mathrm{\sum }_{i=1}^N}{U}_i+{\displaystyle \mathrm{\sum }_{i=1}^N}{V}_i-2Nd\right)/\lambda }\times I[U_{\left(1\right)}\ge d+\theta ,V_{\left(1\right)}\ge d-\theta ]$$

(7)

where $I[·]$ denotes the indicator function. Let the first order statistics $U_{\left(1\right)}$ and $V_{\left(1\right)}$ denote the minimum values of observations ${\left\{U_i\right\}}_{i=1}^N$ and ${\left\{V_i\right\}}_{i=1}^N$, respectively. It can be shown that for fixed values of d and θ, the optimal value of λ that maximizes the likelihood function is given by $\lambda =(\overline{U}+\overline{V})/2-d$. Therefore, the likelihood function Equation (7) can be simplified to:

$$p(U_i,V_i|d,\theta )={\left(\frac{\overline{U}+\overline{V}}{2}-d\right)}^{-2N}{e}^{-2N}\times I[U_{\left(1\right)}\ge d+\theta ,V_{\left(1\right)}\ge d-\theta ]$$

It can be observed that the indicator function poses a high impact on the value of the likelihood function, and moreover, the indicator function depends on $U_{\left(1\right)}$ and $V_{\left(1\right)}$. Hence, in order to maximize the likelihood function, different assumptions on $U_{\left(1\right)}$ and $V_{\left(1\right)}$ are discussed in [28], e.g., $U_{\left(1\right)}>V_{\left(1\right)}>0$, and the region where $p(d,\theta |U_i,V_i)>0$ is illustrated separately for each assumption. The value of θ that maximizes the likelihood function is found graphically from the corresponding support region. It is shown that all of the different assumptions on $U_{\left(1\right)}$ and $V_{\left(1\right)}$ lead to the same MLEs for the vector ${[d,\theta ,\lambda ]}^T$:

$$\left[\begin{array}{c}{\widehat{d}}_{\text{ML}}\\ {\widehat{\theta }}_{\text{ML}}\\ {\widehat{\lambda }}_{\text{ML}}\end{array}\right]=\frac{1}{2}\left[\begin{array}{c}{U}_{\left(1\right)}+V_{\left(1\right)}\\ U_{\left(1\right)}-V_{\left(1\right)}\\ \overline{U}+\overline{V}-U_{\left(1\right)}-V_{\left(1\right)}\end{array}\right]$$

(8)

Our primary interest is the MLE of the clock offset ${\widehat{\theta }}_{\text{ML}}=(U_{\left(1\right)}-V_{\left(1\right)})/2$, but in the meantime, the MLEs for d and λ are also derived. It can be verified that ${\widehat{\theta }}_{\text{ML}}$ is unbiased and has a variance ${\lambda }^2/\left(2N\right)$. Interestingly, if d is unknown and the variable portions of delays are not symmetric, i.e., ${\lambda }_1\ne {\lambda }_2$, the MLE of θ also admits the form ${\widehat{\theta }}_{\text{ML}}=(U_{\left(1\right)}-V_{\left(1\right)})/2$. However, in this case, the MLE is biased.

Different from the graphical analysis used in [28], [29] analytically derived the MLE of clock offset under exponential delays using convex optimization tools. In particular, the system model in Equation (4) is re-written as:

$$U_i=\xi +X_i,V_i=\psi +Y_i$$

(9)

where $\xi \stackrel{\mathrm{\Delta }}{=}d+\theta$ and $\psi \stackrel{\mathrm{\Delta }}{=}d-\theta$. The goal is to determine the MLEs of ξ and ψ, i.e., ${\widehat{\xi }}_{\text{ML}}$ and ${\widehat{\psi }}_{\text{ML}}$, such that the MLE of θ can be obtained from:

$${\widehat{\theta }}_{\text{ML}}=\left({\widehat{\xi }}_{\text{ML}}-{\widehat{\psi }}_{\text{ML}}\right)/2$$

(10)

based on the invariance principle [23]. Towards this end, the density functions of $U_i$ and $V_i$ are expressed as:

$$\begin{array}{ccc}\hfill p\left(U_i\right|\xi )& =& {\lambda }_1^N\text{exp}\left(-{\lambda }_1{\displaystyle \mathrm{\sum }_{i=1}^N}(U_i-\xi )\right)I(U_{\left(1\right)}-\xi )\hfill \end{array}$$

(11)

$$\begin{array}{ccc}\hfill p\left(V_i\right|\psi )& =& {\lambda }_2^N\text{exp}\left(-{\lambda }_2{\displaystyle \mathrm{\sum }_{i=1}^N}(V_i-\psi )\right)I(V_{\left(1\right)}-\psi )\hfill \end{array}$$

(12)

The resulting MLE of ξ can be formulated as a convex optimization problem:

$$\begin{array}{cc}\hfill {\widehat{\xi }}_{\text{ML}}=& arg\underset{\xi }{max}\text{exp}\left(N{\lambda }_1\xi \right)\hfill \\ \hfill \text{s.t.}& \xi \le U_{\left(1\right)}\hfill \end{array}$$

and it follows that ${\widehat{\xi }}_{\text{ML}}=U_{\left(1\right)}.$ The analysis of ψ is analogous to the case of ξ, and it yields that ${\widehat{\psi }}_{\text{ML}}=V_{\left(1\right)}.$ Thus, the MLE of θ follows from Equation (10) and admits the form:

$${\widehat{\theta }}_{\text{ML}}=\frac{U_{\left(1\right)}-V_{\left(1\right)}}{2}$$

which coincides with the result in Equation (8). The advantage of this analytical approach is that it provides a more general derivation of MLEs, which can by applied to obtain the MLEs for the delays under other types of distributions, which include Gaussian and log-normal as special cases [29].

4.1.2. Best Linear Unbiased Estimator

In the statistical signal processing field, the minimum mean square error (MSE) is commonly selected as the criterion to measure the performance of estimators. The MSE of an estimator $\widehat{\theta }$ is defined as:

$$\text{MSE}(\widehat{\theta })=\text{var}(\widehat{\theta })+{[\text{bias}(\widehat{\theta })]}^2$$

(13)

where var$(\widehat{\theta })$ stands for the variance of $\widehat{\theta }$ and bias$(\widehat{\theta })\stackrel{\mathrm{\Delta }}{=}\text{E}(\widehat{\theta })-\theta$.

In general, the MVUE is referred to as the “optimal” estimator, since it minimizes the variance in the class of unbiased estimators. However, it frequently occurs that the MVUE is difficult or impossible to be derived or might not even exist [23]. For such scenarios, a suboptimal estimator is required. A common approach is to constrain the estimator to be linear in terms of the observations and to find the linear estimator that is unbiased and achieves the minimum variance. Such an estimator is referred to as the BLUE. The BLUE of the clock offset under asymmetric exponential delays was first derived by Jeske and Sampath in [30] using the order statistics of the observations ${[U_{\left(1\right)},U_{\left(2\right)},\cdots ,U_{\left(N\right)},V_{\left(1\right)},V_{\left(2\right)},\cdots ,V_{\left(N\right)}]}^T$ where $U_{\left(1\right)}\le U_{\left(2\right)}\le \cdots \le U_{\left(N\right)}$ and $V_{\left(1\right)}\le V_{\left(2\right)}\le \cdots \le V_{\left(N\right)}$. Specifically, the BLUE is formulated as a linear combination of the order statistics:

$$T={\displaystyle \mathrm{\sum }_{i=1}^N}{a}_i{U}_{\left(i\right)}+{\displaystyle \mathrm{\sum }_{i=1}^N}{b}_i{V}_{\left(i\right)}$$

(14)

The variance of estimator T and the unbiasedness condition are further expressed in terms of $a_i$ and $b_i$. Using Lagrange multipliers, it can be shown that the minimum of the variance subject to the unbiasedness condition is achieved when $b_i=-a_i$ with $a_1=1/2+1/\left(2N\right)$ and $a_i=-1/\left[2N(N-1)\right]$ for $2\le i\le N$. Thus, the BLUE of the clock offset admits the form:

$${\widehat{\theta }}_{\text{BLUE-A}}=\frac{N(U_{\left(1\right)}-V_{\left(1\right)})-(\overline{U}-\overline{V})}{2(N-1)}$$

(15)

On the other hand, the BLUE of the clock offset under symmetric exponential delays can also be derived following the aforementioned steps, and it is expressed as:

$${\widehat{\theta }}_{\text{BLUE-S}}=\frac{U_{\left(1\right)}-V_{\left(1\right)}}{2}$$

Using statistical signal processing techniques, [31] later re-derived the joint BLUEs for the fixed delay, the clock offset and the mean of exponential delays under symmetric and asymmetric exponential delays, as shown in Equations (16) and (17), respectively.

$$\left[\begin{array}{c}{\widehat{d}}_{\text{BLUE-S}}\\ {\widehat{\theta }}_{\text{BLUE-S}}\\ {\widehat{\lambda }}_{\text{BLUE-S}}\end{array}\right]=\frac{1}{2(N-1)}\left[\begin{array}{c}N(U_{\left(1\right)}+V_{\left(1\right)})-(\overline{U}+\overline{V})\\ (N-1)(U_{\left(1\right)}-V_{\left(1\right)})\\ N(\overline{U}+\overline{V})-N(U_{\left(1\right)}+V_{\left(1\right)})\end{array}\right]$$

(16)

$$\left[\begin{array}{c}{\widehat{d}}_{\text{BLUE-A}}\\ {\widehat{\theta }}_{\text{BLUE-A}}\\ {\widehat{\lambda }}_{1\text{BLUE-A}}\\ {\widehat{\lambda }}_{2\text{BLUE-A}}\end{array}\right]=\frac{1}{2(N-1)}\left[\begin{array}{c}N(U_{\left(1\right)}+V_{\left(1\right)})-(\overline{U}+\overline{V})\\ N(U_{\left(1\right)}-V_{\left(1\right)})-(\overline{U}-\overline{V})\\ 2N(\overline{U}-U_{\left(1\right)})\\ 2N(\overline{V}-V_{\left(1\right)})\end{array}\right]$$

(17)

These results were further used as a prerequisite to find the MVUE of the clock offset, a result that will be presented in the next sub-section.

4.1.3. Minimum Variance Unbiased Estimator

As discussed earlier, the MVUE is an unbiased estimator that exhibits lower variance than any other unbiased estimator. Finding the MVUE usually requires usage of the concept of sufficient statistics, the Neyman–Fisher factorization theorem and the Rao–Blackwell–Lehmann–Scheffe theorem [23] (p. 101).

Let Φ denote the parameters to be estimated and z represent the set of the observations; then, the Neyman–Fisher factorization theorem states that if the likelihood function $p\left(\mathbf{z}\right|\mathbf{\Phi })$ can be factorized as:

$$p\left(\mathbf{z}\right|\mathbf{\Phi })=g(T\left(\mathbf{z}\right),\mathbf{\Phi }\left)h\right(\mathbf{z}),$$

where g is a function depending on z only through $T\left(\mathbf{z}\right)$ and h is a function depending on z only, then $T\left(\mathbf{z}\right)$ is a sufficient statistic for Φ. In a two-way message exchange mechanism under symmetric exponential delays, $\mathbf{\Phi }={[d,\theta ,\lambda ]}^T$ and $\mathbf{z}={[U_1,U_2,\cdots ,U_N,V_1,V_2,\cdots ,V_N]}^T$. The MVUE of Φ was derived in [31] by implementing the following procedure. Express first the likelihood function in terms of the unit step function $u[·]$ as follows:

$$p\left(\mathbf{z}\right|\mathbf{\Phi })={\lambda }^{-2N}\text{exp}\left[-\frac{1}{\lambda }{\displaystyle \mathrm{\sum }_{i=1}^N}(U_i+V_i-2d)\right]·u[U_{\left(1\right)}-d-\theta ]·u[V_{\left(1\right)}-d+\theta ]$$

(18)

and factorize Equation (18) as:

$$p\left(\mathbf{z}\right|\mathbf{\Phi })=g_1({\displaystyle \mathrm{\sum }_{i=1}^N}(U_i+V_i),d,\lambda )·g_2(U_{\left(1\right)},d,\theta )·g_3(V_{\left(1\right)},d,\theta )·h\left(\mathbf{z}\right)$$

where:

$$g_1({\displaystyle \mathrm{\sum }_{i=1}^N}(U_i+V_i),d,\lambda )={\lambda }^{-2N}\text{exp}\left[-\frac{1}{\lambda }{\displaystyle \mathrm{\sum }_{i=1}^N}(U_i+V_i-2d)\right]$$

$$g_2(U_{\left(1\right)},d,\theta )=u[U_{\left(1\right)}-d-\theta ],g_3(V_{\left(1\right)},d,\theta )=u[V_{\left(1\right)}-d+\theta ],h\left(\mathbf{z}\right)=1$$

In the above equations, $h\left(\mathbf{z}\right)$ is independent of the unknown parameters Φ and $g_1,g_2,g_3$ are functions depending on the data only through $\mathbf{T}=\{{\displaystyle \mathrm{\sum }_{i=1}^n}(U_i+V_i),U_{\left(1\right)},V_{\left(1\right)}\}$. Using the Neyman–Fisher factorization theorem, it turns out that T is a sufficient statistic. Moreover, the Rao–Blackwell–Lehmann–Scheffe theorem (p. 109 in [23]) claims that a sufficient statistic T is complete if there is only one function $c(·)$ of T that is unbiased, and this function leads to the MVUE, i.e., ${\mathbf{\Phi }}_{\text{MVU}}=c\left(\mathbf{T}\right)$. Therefore, the remaining task is to prove that T is complete or, equivalently, that only one function of T is unbiased, and to find that function.

The authors in [31] employed a one-to-one transformation of T, denoted as ${\mathbf{T}}^{\prime }=\{{\displaystyle \mathrm{\sum }_{i=1}^N}(U_i+V_i-U_{\left(1\right)}-V_{\left(1\right)}),U_{\left(1\right)},V_{\left(1\right)}\}$. Then, it was shown that ${\mathbf{T}}^{\prime }$ is complete by assuming that there are two functions of ${\mathbf{T}}^{\prime }$ leading to unbiasedness, i.e., $\text{E}\left(c\left({\mathbf{T}}^{\prime }\right)\right)=\text{E}\left(h\left({\mathbf{T}}^{\prime }\right)\right)=\mathbf{\Phi }$, and then, proving that, actually, $c\left({\mathbf{T}}^{\prime }\right)=h\left({\mathbf{T}}^{\prime }\right)$. It turns out that T is also complete, since the sufficient statistics are unique within one-to-one transformations [23]. To this end, what remains to prove is to find an unbiased estimator for Φ as a function of T, which represents the MVUE according to the Rao–Blackwell–Lehmann–Scheffe theorem. It seems difficult to find three unbiased functions of T for each of $d,\theta$ and λ just by inspection. However, it can be observed that the BLUE ${\mathbf{\Phi }}_{\text{BLUE-S}}$ in Equation (16) is a function of T and is also unbiased. Therefore, it is concluded that the BLUE is also the MVUE:

$${\widehat{\mathbf{\Phi }}}_{\text{MVUE-S}}=\left[\begin{array}{c}{\widehat{d}}_{\text{MVUE-S}}\\ {\widehat{\theta }}_{\text{MVUE-S}}\\ {\widehat{\lambda }}_{\text{MVUE-S}}\end{array}\right]=\frac{1}{2(N-1)}\left[\begin{array}{c}N(U_{\left(1\right)}+V_{\left(1\right)})-(\overline{U}+\overline{V})\\ (N-1)(U_{\left(1\right)}-V_{\left(1\right)})\\ N(\overline{U}+\overline{V})-N(U_{\left(1\right)}+V_{\left(1\right)})\end{array}\right]$$

(19)

Following an analogous approach, it was reported in [31] that the MVUE of the clock offset is also equivalent to the BLUE in the presence of asymmetric exponential delays:

$${\widehat{\theta }}_{\text{MVUE-A}}=\frac{N(U_{\left(1\right)}-V_{\left(1\right)})-(\overline{U}-\overline{V})}{2(N-1)}$$

4.1.4. Comparison of Estimators

A comparison of MLE, BLUE and MVUE under both symmetric and asymmetric exponential delays is shown in Table 1. The criteria adopted in Table 1 include the estimator formula in terms of observations, bias, variance and MSE. It is observed that the BLUE and MVUE coincide in the presence of both symmetric and asymmetric distributions. Furthermore, all three estimators admit the same expression when the random delays are symmetrically exponential distributed. Under asymmetric exponential delays, albeit unbiased, the MLE can still outperform the MVUE when:

$$\frac{{\lambda }_1^2+{\lambda }_2^2-{\lambda }_1{\lambda }_2}{2N^2}<\frac{{\lambda }_1^2+{\lambda }_2^2}{4N(N-1)}$$

or equivalently:

$$N<\frac{2{\lambda }_1{\lambda }_2}{{({\lambda }_1-{\lambda }_2)}^2}+2$$

(20)

It can be seen that the MLE achieves a better performance when the means of the exponential delays are close to each other. In this way, the right-hand side of Equation (20) resumes being a large number, and the inequality Equation (20) generally holds with a reasonable selection of N.

Table 1. Comparison of estimators.

Clock Offset	Symmetric Delays				Asymmetric Delays
	Formula	Bias	Variance	MSE	Formula	Bias	Variance	MSE
MLE [28,29]	$\frac{U_{\left(1\right)}-V_{\left(1\right)}}{2}$	0	$\frac{{\lambda }^2}{2N^2}$	$\frac{{\lambda }^2}{2N^2}$	$\frac{U_{\left(1\right)}-V_{\left(1\right)}}{2}$	$\frac{{\lambda }_1-{\lambda }_2}{2N}$	$\frac{{\lambda }_1^2+{\lambda }_2^2}{4N^2}$	$\frac{{\lambda }_1^2+{\lambda }_2^2-{\lambda }_1{\lambda }_2}{2N^2}$
BLUE [30,31]	$\frac{U_{\left(1\right)}-V_{\left(1\right)}}{2}$	0	$\frac{{\lambda }^2}{2N^2}$	$\frac{{\lambda }^2}{2N^2}$	$\frac{N(U_{\left(1\right)}-V_{\left(1\right)})-(\overline{U}-\overline{V})}{2(N-1)}$	0	$\frac{{\lambda }_1^2+{\lambda }_2^2}{4N(N-1)}$	$\frac{{\lambda }_1^2+{\lambda }_2^2}{4N(N-1)}$
MVUE [31]	$\frac{U_{\left(1\right)}-V_{\left(1\right)}}{2}$	0	$\frac{{\lambda }^2}{2N^2}$	$\frac{{\lambda }^2}{2N^2}$	$\frac{N(U_{\left(1\right)}-V_{\left(1\right)})-(\overline{U}-\overline{V})}{2(N-1)}$	0	$\frac{{\lambda }_1^2+{\lambda }_2^2}{4N(N-1)}$	$\frac{{\lambda }_1^2+{\lambda }_2^2}{4N(N-1)}$

Table 1. Comparison of estimators.

Clock Offset	Symmetric Delays				Asymmetric Delays
	Formula	Bias	Variance	MSE	Formula	Bias	Variance	MSE
MLE [28,29]	$\frac{U_{\left(1\right)}-V_{\left(1\right)}}{2}$	0	$\frac{{\lambda }^2}{2N^2}$	$\frac{{\lambda }^2}{2N^2}$	$\frac{U_{\left(1\right)}-V_{\left(1\right)}}{2}$	$\frac{{\lambda }_1-{\lambda }_2}{2N}$	$\frac{{\lambda }_1^2+{\lambda }_2^2}{4N^2}$	$\frac{{\lambda }_1^2+{\lambda }_2^2-{\lambda }_1{\lambda }_2}{2N^2}$
BLUE [30,31]	$\frac{U_{\left(1\right)}-V_{\left(1\right)}}{2}$	0	$\frac{{\lambda }^2}{2N^2}$	$\frac{{\lambda }^2}{2N^2}$	$\frac{N(U_{\left(1\right)}-V_{\left(1\right)})-(\overline{U}-\overline{V})}{2(N-1)}$	0	$\frac{{\lambda }_1^2+{\lambda }_2^2}{4N(N-1)}$	$\frac{{\lambda }_1^2+{\lambda }_2^2}{4N(N-1)}$
MVUE [31]	$\frac{U_{\left(1\right)}-V_{\left(1\right)}}{2}$	0	$\frac{{\lambda }^2}{2N^2}$	$\frac{{\lambda }^2}{2N^2}$	$\frac{N(U_{\left(1\right)}-V_{\left(1\right)})-(\overline{U}-\overline{V})}{2(N-1)}$	0	$\frac{{\lambda }_1^2+{\lambda }_2^2}{4N(N-1)}$	$\frac{{\lambda }_1^2+{\lambda }_2^2}{4N(N-1)}$

4.2. Joint Estimation of Clock Offset and Skew under Exponential Delays

In general, the clock synchronization in WSNs involves two steps. The first step assumes synchronizing the nodes by adjusting the clock phase offset (close offset), while the second step resumes correcting the clock frequency offset (clock skew). The clock skew correction plays a critical role in clock synchronization, especially for long-term synchronization, since it reduces the number of exchanged messages and, thus, brings down the energy consumption. Therefore, we discuss the joint estimation of clock offset and skew in this section.

In the literature, based on a two-way message exchange mechanism, the studies for joint estimation of clock offset and skew mainly concentrated on the development of MLEs, and there is no closed-form solution for the joint estimation of clock offset and skew under exponential delays. Roughly speaking, the algorithms for developing MLEs can be categorized into two types: in the first category, a suboptimal model is built by removing some nuisance parameters, and the corresponding MLEs are derived, while in the second category, the joint estimation is conducted directly.

4.2.1. Removing Nuisance Parameters

Based on the two-way message exchange mechanism in Equation (2) and assuming symmetric exponential delays, the likelihood function based on observations ${\left\{U_i\right\}}_{i=1}^N$ and ${\left\{V_i\right\}}_{i=1}^N$ is given by:

$$\begin{array}{cc}\hfill p\left({\mathbf{T}}_i\right|d,\theta ,f,\lambda )=& {\lambda }^{-2N}\text{exp}\left[-\frac{1}{\lambda }\left({\displaystyle \mathrm{\sum }_{i=1}^N}\frac{T_i^b-T_i^c}{f}-{\displaystyle \mathrm{\sum }_{i=1}^N}(T_i^a-T_i^d)-2Nd\right)\right]\hfill \\ & \times {\displaystyle \prod _{i=1}^N}I\left[\frac{T_i^b-\theta }{f}-T_i^a-d\ge 0;T_i^d-\frac{T_i^c-\theta }{f}-d\ge 0\right]\hfill \end{array}$$

(21)

It is observed that the likelihood function Equation (21) assumes a complicated expression, and the MLEs might be difficult to derive. According to this observation, an algorithm for joint estimation of clock offset and skew, referred to as the ML-like estimator (MLL), was proposed in [18] by using the first and last observations of timing message exchanges. Specifically, from the first equation in Equation (2), subtracting $T_1^b$ from $T_N^b$ yields:

$$T_N^b-T_1^b=f·(T_N^a-T_1^a+X_N-X_1)$$

(22)

Similarly, from the second equation in Equation (2), subtracting $T_1^c$ from $T_N^c$ leads to:

$$T_N^c-T_1^c=f·(T_N^d-T_1^d+Y_1-Y_N)$$

(23)

Such an approach helps to remove the explicit dependency on the nuisance parameters λ, d and θ. In addition, an ML-like estimate ${\widehat{f}}_{\text{MLL}}$ of the clock skew is derived using Equation (22) and (23). Finding an estimate of the clock offset is achieved by rewriting Equation (2) as:

$$\begin{array}{cc}\hfill U_i^{\prime }=& d^{\prime }+\theta +X_i^{\prime }\hfill \\ \hfill V_i^{\prime }=& d^{\prime }-\theta +Y_i^{\prime }\hfill \end{array}$$

where $U_i^{\prime }=T_i^b-T_i^a{\widehat{f}}_{\text{MLL}},V_i^{\prime }=T_i^d{\widehat{f}}_{\text{MLL}}-T_i^c,X_i^{\prime }=X_i{\widehat{f}}_{\text{MLL}},Y_i^{\prime }=Y_i{\widehat{f}}_{\text{MLL}},$ and $d^{\prime }=d{\widehat{f}}_{\text{MLL}}$. It can be seen that the above model shares the same form as the no-skew model in Equation (4). Therefore, an ML-like estimate of the clock offset is obtained ${\widehat{\theta }}_{\text{MLL}}=(U_{\left(1\right)}^{\prime }-V_{\left(1\right)}^{\prime })/2$. The above algorithm for joint estimation of clock offset and skew is computationally simple, but it exhibits a suboptimal performance compared to the maximum likelihood estimator, since it exploits a reduced set of statistics. That is the reason why the above estimator is referred to as the ML-like estimator.

Another approach to remove the nuisance parameter was reported in [32] by adding two equations in Equation (2). It follows that:

$$T_i^b+T_i^c=f(T_i^a+T_i^d)+2\theta +f(X_i-Y_i)$$

(24)

where the nuisance parameter d is removed. Dividing the above expression by f and defining ${\theta }_1=1/f$ and ${\theta }_2=\theta /f$ leads to a new model represented in matrix form as follows:

$$\left[\begin{array}{c}{T}_1^a+T_1^d\\ ⋮\\ T_N^a+T_N^d\end{array}\right]=\left[\begin{array}{cc}{T}_1^b+T_1^c& -2\\ ⋮& ⋮\\ T_N^b+T_N^c& -2\end{array}\right]\left[\begin{array}{c}{\theta }_1\\ {\theta }_2\end{array}\right]+\left[\begin{array}{c}{Y}_1-X_1\\ ⋮\\ Y_N-X_N\end{array}\right]$$

(25)

Assuming symmetric exponential delays, it is easy to verify that $Z_i\stackrel{\mathrm{\Delta }}{=}{X}_i-Y_i$ follows a Laplace distribution with location parameter zero and scale parameter $1/\lambda$. Thus, the log-likelihood function for model Equation (25) is expressed as:

$$\text{ln} p\left({\mathbf{T}}_i\right|{\theta }_1,{\theta }_2)=N \text{ln}\frac{\lambda }{2}-\lambda {\displaystyle \mathrm{\sum }_{i=1}^N}|T_i^a+T_i^d-{\theta }_1(T_i^b+T_i^c)+2{\theta }_2|$$

(26)

Therefore, the MLEs of ${\theta }_1$ and ${\theta }_2$ can be found by maximizing Equation (26), and the corresponding estimators for θ and f are obtained using the relationships ${\theta }_1=1/f$ and ${\theta }_2=\theta /f$.

4.2.2. Direct Joint Estimation of Clock Offset and Skew

Assuming symmetric exponential delays with a known mean λ, [33] addressed the problem of MLEs for clock offset and skew by optimizing over nonlinear constraints iteratively. In terms of the estimates ${[d,\theta ,f]}^T$, four different cases were considered: (1) d and θ known, f unknown; (2) θ known, d and f unknown; (3) d known, θ and f unknown; (4) $d,\theta ,f$ unknown. The ultimate goal is to jointly estimate ${[d,\theta ,f]}^T$ in the fourth case. However, similar to the derivation of the MLE in [28] when no skew is considered, the MLEs are also determined by exploiting the boundary conditions of the support regions. For Cases 1 and 2, the support regions are 2D, and the discussion of these two cases gives an insight to further analyze Cases 3 and 4, where the support regions admit a 3D visualization.

Li and Jeske [34] proposed a computationally-simpler algorithm to find the MLEs of clock offset and skew. Asymmetric exponential delays with unknown means were assumed in [34], such that the estimation algorithm can be applied to a more general framework. In this case, the likelihood function resumes as:

$$\begin{array}{ll}p\left({\mathbf{T}}_i\right|d,\theta ,f,{\lambda }_1,{\lambda }_2)=\hfill & \frac{{\lambda }_1^N{\lambda }_2^N}{f^{2N}}{\displaystyle \prod _{i=1}^N}\text{exp}\left[-\frac{{\lambda }_1}{f}(T_i^b-f{T}_i^a-\theta -fd)-\frac{{\lambda }_2}{f}(f{T}_i^d-T_i^c+\theta -fd)\right]\hfill \\ & \times I[min(T_i^b-f{T}_i^a-\theta -fd)\ge 0;min(f{T}_i^d-T_i^c+\theta -fd)\ge 0]\hfill \end{array}$$

(27)

Re-parameterizing in terms of ${\theta }_1=\theta +fd$ and ${\theta }_2=-\theta +fd$ leads to:

$$\begin{array}{cc}\hfill p\left({\mathbf{T}}_i\right|f,{\theta }_1,{\theta }_2,{\lambda }_1,{\lambda }_2)=& \frac{{\lambda }_1^N{\lambda }_2^N}{f^{2N}}{\displaystyle \prod _{i=1}^N}\text{exp}\left[-\frac{{\lambda }_1}{f}(T_i^b-f{T}_i^a-{\theta }_1)-\frac{{\lambda }_2}{f}(f{T}_i^d-T_i^c-{\theta }_2)\right]\hfill \\ & \times I[min(T_i^b-f{T}_i^a-{\theta }_1)\ge 0;min(f{T}_i^d-T_i^c-{\theta }_2)\ge 0]\hfill \end{array}$$

(28)

To find the MLEs for likelihood function Equation (27), or equivalently Equation (28), the concept of profile likelihood function is employed in [34], such that the MLEs can be alternatively found by maximizing the profile likelihoods. In particular, fixing $f,{\theta }_1$ and ${\theta }_2$, the conditional MLEs of ${\lambda }_1$ and ${\lambda }_2$ are shown to be:

$${\widehat{\lambda }}_1(f,{\theta }_1,{\theta }_2)=\frac{fN}{{\displaystyle \mathrm{\sum }_{i=1}^n}(T_i^b-f{T}_i^a-{\theta }_1)},{\widehat{\lambda }}_2(f,{\theta }_1,{\theta }_2)=\frac{fN}{{\displaystyle \mathrm{\sum }_{i=1}^n}(T_i^d-f{T}_i^c-{\theta }_2)}$$

Plugging the above equations into Equation (28) yields the profile likelihood as a function of $f,{\theta }_1$ and ${\theta }_2$, which is denoted as $L_p(f,{\theta }_1,{\theta }_2)$. Following the same procedure with respect to $L_p(f,{\theta }_1,{\theta }_2)$, the conditional MLEs for ${\theta }_1$ and ${\theta }_2$ are expressed as:

$${\widehat{\theta }}_1\left(f\right)=min(T_i^b-f{T}_i^a),{\widehat{\theta }}_2\left(f\right)=min(f{T}_i^d-T_i^c)$$

To this end, the original five-dimensional optimization problem Equation (27) is simplified to the maximization of a one-dimensional function $L_p(f,min(T_i^b-f{T}_i^a),min(f{T}_i^d-T_i^c))$, whose solution represents the MLE for clock skew, and it is denoted as ${\widehat{f}}_{JML}$. Additionally, the MLE for clock offset is obtained via:

$${\widehat{\theta }}_{JML}=\frac{min(T_i^b-{\widehat{f}}_{JML}{T}_i^a)-min({\widehat{f}}_{JML}{T}_i^d-T_i^c)}{2}$$

since the corresponding estimates of ${\theta }_1$ and ${\theta }_2$ are $min(T_i^b-{\widehat{f}}_{JML}{T}_i^a)$ and $min({\widehat{f}}_{JML}{T}_i^d-T_i^c)$, respectively.

4.3. Confidence Interval for Clock Offset

From a statistical point of view, a confidence interval (CI) is used to describe the amount of uncertainty associated with a sample estimate of a parameter. More specifically, if a CI is constructed among a series of experiments, the proportion of such an interval that contains the true value of the parameter is given by the confidence level. Mathematically, suppose X to be a random sample from a probability distribution with parameter θ, a CI is an interval with endpoints $\left\{s\right(X),b(X\left)\right\}$, such that the following property holds:

$$\text{Pr}\left(s\right(X)<\theta <b(X\left)\right)=\gamma$$

(29)

where γ stands for the confidence level.

In terms of the confidence interval for clock offset under exponential delays, Li et al. [35] derived satisfactory CIs using an approximate pivotal quantity (APQ). By assuming asymmetric exponential delays, the authors in [35] derived the relative likelihood function for θ, say $\mathrm{\Lambda }\left(\theta \right)$, based on the likelihood function from model Equation (4). It was then shown that $\mathrm{\Lambda }\left(\theta \right)$ is an asymptotic pivot in the sense that its limiting distribution does not depend on any unknown parameter. Thus, the asymptotic pivot property of $\mathrm{\Lambda }\left(\theta \right)$, as well as its easily invertible form lead to a way to obtain an asymptotic $100(1-\alpha )\%$ CI for θ as follows:

$$\begin{array}{c}\hfill s(U_i,V_i)=max\left\{\frac{U_{\left(1\right)}-V_{\left(1\right)}}{2}-\frac{\overline{U}-U_{\left(1\right)}}{2}\left(\frac{1}{{\alpha }^{\frac{1}{N}}}-1\right),-V_{\left(1\right)}\right\}\\ \hfill b(U_i,V_i)=min\left\{\frac{U_{\left(1\right)}-V_{\left(1\right)}}{2}+\frac{\overline{V}-V_{\left(1\right)}}{2}\left(\frac{1}{{\alpha }^{\frac{1}{N}}}-1\right),-U_{\left(1\right)}\right\}\end{array}$$

(30)

A generalized pivotal quantity (GPQ) was also employed in [35] to propose a generalized confidence interval for θ. Even though generalized confidence intervals generally do not have frequentist coverage probabilities that are exactly equal to the nominal value, the proposed GPQ actually leads to frequentist coverage probabilities that are very close to the nominal value. Furthermore, APQ and GPQ CIs under exponential delays were also extended in [35] to any distribution within the one-parameter scale family.

Further discussions about CIs of clock offset were carried out in [36,37]. Specifically, under exponential network delays, [36] explored sequential intervals with a preset width and proved that these CIs provide a satisfactory solution to minimize the number of samples. In this way, sequential CIs can be used to obtain the smallest sample size required to achieve a specified precision for the clock offset estimation. On the other hand, by considering the correlation between the uplink and downlink random delays, [37] extended the CI procedure to bivariate models where the random delays are formulated as Freund and Marshall–Olkin bivariate exponential delays. In addition, the MLEs for clock offset under Freund and Marshall–Olkin models were also derived.

5. Pairwise Clock Synchronization under Unknown Random Delays

Although the assumptions of Gaussian and exponential distributions for the random delay component of WSNs are plausible, it might happen that the underlying PDF of the network random delays is unknown in advance. In fact, it is well known that the distribution of the network random delays is difficult to characterize succinctly [30]. Therefore, the performance of estimators particularly designed for a certain distribution may degenerate greatly under another type of distribution, and there is a need for developing efficient estimation methods that are robust to the unknown random delays of WSNs. Based on the classic two-way message exchange mechanism, we describe several robust estimation methods in this section, namely the bootstrap bias correction [30], the composite particle filtering [38] and the least squares estimator [39].

5.1. Bootstrap Bias Correction

Bootstrap bias correction is a statistical approach that aims to reduce the bias of an estimator at the expense of increased variance, but with a total effect of reduced MSE [40]. For example, the MLE of clock offset derived under exponential delays is given by $\widehat{\theta }=(U_{\left(1\right)}-V_{\left(1\right)})/2$. However, the random delays may not follow an exponential distribution exactly, and the MLE derived under the exponential assumption may perform poorly under other distributions. For such scenarios, the bootstrap bias correction approach can be applied to reduce the bias and eventually the MSE of the estimator.

Suppose the observations x follow an unknown distribution F and the estimator is a statistic $\widehat{\theta }=s\left(x\right)$; the bias of $\widehat{\theta }$ is defined as $B(\widehat{\theta })=E_F\left[s\left(x\right)\right]-\theta$. The bootstrap estimate of the bias can be expressed as:

$$\widehat{B}(\widehat{\theta })=E_{\widehat{F}}\left[s\left(x\right)\right]-\widehat{\theta }$$

(31)

and the bias corrected estimator admits the form:

$${\widehat{\theta }}_{BC}=\widehat{\theta }-\widehat{B}(\widehat{\theta })$$

(32)

where $\widehat{F}$ denotes the empirical distribution of observations.

In the context of clock offset estimation under exponential delays, the observations between two nodes are denoted as ${\left\{U_i\right\}}_{i=1}^N$ and ${\left\{V_i\right\}}_{i=1}^N$, and the MLE is given by $\widehat{\theta }=(U_{\left(1\right)}-V_{\left(1\right)})/2$. The bias of $\widehat{\theta }$ resumes as:

$$B(\widehat{\theta })=E_F\left(\frac{U_{\left(1\right)}-V_{\left(1\right)}}{2}\right)-\theta =\frac{1}{2}\left({\int }_0^{\infty }{[1-F\left(u\right)]}^{N}du-{\int }_0^{\infty }{[1-G\left(v\right)]}^{N}dv\right)-\theta$$

where $F\left(u\right)$ and $G\left(v\right)$ stand for the unknown distributions of observations ${\left\{U_i\right\}}_{i=1}^N$ and ${\left\{V_i\right\}}_{i=1}^N$, respectively. The bootstrap estimate of $B(\widehat{\theta })$ can be further expressed as:

$$\widehat{B}(\widehat{\theta })=E_{\widehat{F}}\left(\frac{U_{\left(1\right)}-V_{\left(1\right)}}{2}\right)-\widehat{\theta }=\frac{1}{2}\left({\int }_0^{\infty }{[1-\widehat{F}\left(u\right)]}^{N}du-{\int }_0^{\infty }{[1-\widehat{G}\left(v\right)]}^{N}dv\right)-\widehat{\theta }$$

(33)

Defining $U_{\left(0\right)}=V_{\left(0\right)}=0$ and $U_{(N+1)}=V_{(N+1)}=\infty$ leads to the empirical distribution [30]:

$$\begin{array}{cc}\hfill 1-\widehat{F}\left(u\right)& ={\displaystyle \mathrm{\sum }_{i=1}^{N+1}}\frac{N-i+1}{N}I[U_{(i-1)}\le u<U_{\left(i\right)}]\hfill \\ \hfill 1-\widehat{G}\left(u\right)& ={\displaystyle \mathrm{\sum }_{i=1}^{N+1}}\frac{N-i+1}{N}I[V_{(i-1)}\le v<V_{\left(i\right)}]\hfill \end{array}$$

(34)

Plugging Equation (34) back into Equation (33) yields $\widehat{B}(\widehat{\theta })$ and eventually the bias-corrected estimator of $\widehat{\theta }$:

$${\widehat{\theta }}_{BC}=U_{\left(1\right)}-V_{\left(1\right)}-\frac{1}{2}{\displaystyle \mathrm{\sum }_{i=1}^N}\left[{\left(\frac{N-i+1}{N}\right)}^N-{\left(\frac{N-i}{N}\right)}^N\right]\left[U_{\left(i\right)}-V_{\left(i\right)}\right]$$

(35)

Following the same procedure described above, the bias-corrected estimator of the BLUE under asymmetric exponential delays, i.e., Equation (15), was also derived in [30]. Experimental results in [30] showed that under common distribution assumptions other than exponential, e.g., Gamma, Weibull and log-normal, the bias-corrected estimator of BLUE has the smallest MSE, while ${\widehat{\theta }}_{\text{BC}}$ also outperforms the MLE and the BLUE derived for exponential delays.

The work in [41] investigated the BLUE and its corresponding bias-corrected estimator under a Pareto distribution: a heavy-tailed distribution that was adopted to model network delays for recent applications [42,43,44]. The authors in [41] examined the effectiveness of bootstrap bias correction of different estimators under varying assumptions for network delays. Some interesting examples were reported where bootstrap bias correction fails in the sense that the MSE increases or even the absolute bias increases. For instance, a somewhat surprising result is that the bootstrap bias correction of the exponential BLUE leads to a large absolute bias under Pareto network delays. Among these estimators and their corresponding bootstrap bias corrections, the bias-corrected estimator in Equation (35) was considered to be a significantly more robust estimator in [41].

Moreover, another bias correction method, referred to as the Jackknifebias-correction, was reported in [45]. The Jackknife bias-corrected estimator of the exponential MLE, $\widehat{\theta }=(U_{\left(1\right)}-V_{\left(1\right)})/2$, was proposed and compared to the corresponding bootstrap bias-corrected estimator Equation (35). Simulation results in [45] illustrated that the Jackknife bias correction does a better job at reducing bias, but is generally eroded by a variance increase, which yields a larger MSE.

5.2. Composite Particle Filtering

The idea of composite particle filtering is to model the estimation problem in the form of a state-space representation and to exploit an optimal Bayesian approach for state estimation [9]. In terms of clock offset estimation, the system model in Equation (4) is rewritten in compact form matrix representation:

$$\underset{\stackrel{\mathrm{\Delta }}{=}{\mathbf{y}}_i}{\underbrace{\left[\begin{array}{c}{U}_i\\ V_i\end{array}\right]}}=\underset{\stackrel{\mathrm{\Delta }}{=}\mathbf{B}}{\underbrace{\left[\begin{array}{cc}1& 1\\ 1& -1\end{array}\right]}}\underset{\stackrel{\mathrm{\Delta }}{=}{\mathbf{x}}_i}{\underbrace{\left[\begin{array}{c}d\\ \theta \end{array}\right]}}+\underset{\stackrel{\mathrm{\Delta }}{=}{\mathit{\omega }}_i}{\underbrace{\left[\begin{array}{c}{X}_i\\ Y_i\end{array}\right]}}$$

(36)

where ${\omega }_i$ can assume any distribution. Since the value of clock offset is time varying, it is assumed to obey a Gauss-Markov dynamic model:

$${\mathrm{x}}_i={\mathrm{x}}_{i-1}+{\nu }_{i-1}$$

(37)

where the noise term ${\nu }_{i-1}$ is modeled as a Gaussian random variable with zero mean and covariance matrix ${\mathrm{Q}}_{k-1}=E\left[{\nu }_{i-1}{\nu }_{i-1}^T\right]$. The aim is to obtain the minimum mean square error (MSEE) estimator of state ${\mathrm{x}}_i$, which is expressed as the conditional mean state estimator:

$$\widehat{{\mathrm{x}}_i}=E\left[{\mathrm{x}}_i\right|{\mathrm{y}}_{1:i}]$$

(38)

where $y_{1:i}={[y_1,\cdots ,y_i]}^T$ stands for the observations up to time i.

The optimal state estimator consists of two steps. In the first step, the posterior distribution $p\left({\mathrm{x}}_{i-1}\right|{\mathrm{y}}_{1:i-1})$ is updated and supposed known, then the predictive distribution is updated as follows:

$$p\left({\mathrm{x}}_i\right|{\mathrm{y}}_{1:i-1})=\int p\left({\mathrm{x}}_i\right|{\mathrm{x}}_{i-1})p\left({\mathrm{x}}_{i-1}\right|{\mathrm{y}}_{1:i-1})d{\mathrm{x}}_{i-1}$$

(39)

where the transition PDF is obtained from Equation (37). In the second step, the posterior distribution at time i is evaluated:

$$p\left({\mathrm{x}}_i\right|{\mathrm{y}}_{1:i})=C_i·p\left({\mathrm{x}}_i\right|{\mathrm{y}}_{1:i-1})p\left({\mathrm{y}}_i\right|{\mathrm{x}}_i)$$

(40)

where $p\left({\mathrm{y}}_i\right|{\mathrm{x}}_i)$ represents the likelihood function obtained from Equation (36), and:

$$C_i={\left(\int p\left({\mathrm{x}}_i\right|{\mathrm{y}}_{1:i})p\left({\mathrm{y}}_i\right|{\mathrm{x}}_i)d{\mathrm{x}}_i\right)}^{-1}$$

is a normalization constant.

In the composite particle filtering algorithm adopted in [38], the posterior and predictive distributions are recursively modeled by finite Gaussian mixtures, and the components of the Gaussian mixtures are updated using the Kalman filter in conjunction with particle filters. Computer simulation results illustrate that the performance of the composite particle filter algorithm is superior in the presence of delays with unknown distribution (such as Gamma, Weibull, log-normal or arbitrary mixtures) relative to the performance exhibited by the MLEs derived specifically under the assumption of Gaussian or exponential delays.

5.3. Least Squares Estimators

The least squares approach in [39] provides a general framework for joint estimation of clock offset and skew, irrespective of the network delay distribution type. The model adopted by the least squares approach is obtained by adding two equations in Equation (2) together and reduces to:

$$\frac{T_i^b+T_i^c}{2}=\theta +f\frac{T_i^a+T_i^d}{2}+f\frac{X_i-Y_i}{2}$$

(41)

Thus, we can regard Equation (41) as a linear regression model in terms of $\{(T_i^b+T_i^c)/2$, $(T_i^a+T_i^d){/2\}}_{i=1}^N$, where θ represents the intercept, f stands for the slope and $f(X_i-Y_i)/2$ denotes the error term. Based on the model Equation (41), θ and f can be naively estimated via a standard least squares approach as follows:

$${\widehat{f}}_{\text{LS}}=\frac{{\displaystyle \mathrm{\sum }_{i=1}^N}(T_i^b+T_i^c)(T_i^a+T_i^d-{\overline{T}}^a-{\overline{T}}^d)}{{\displaystyle \mathrm{\sum }_{i=1}^N}{(T_i^a+T_i^d-{\overline{T}}^a-{\overline{T}}^d)}^2}$$

(42)

and:

$${\widehat{\theta }}_{\text{LS}}=({\overline{T}}^b+{\overline{T}}^c)/2-{\widehat{f}}_{\text{LS}}({\overline{T}}^a+{\overline{T}}^d)/2$$

(43)

where ${\overline{T}}^j$ for $j=a,b,c,d$ represent sample means.

However, in general, neither the mean nor the covariance of the error term $f(X_i-Y_i)/2$ is zero, which may result in poor estimation performance. Therefore, several alternative least squares estimators were proposed in [39] to further improve the estimation accuracy of clock offset and skew. An example of an enhanced least squares estimator is referred to as the constrained least squares (CLS) estimator, and it is found by minimizing the sum of squared residuals subject to a set of constraints on θ and f formed by considering the time stamp model in Equation (2):

$$T_i^b\ge \theta +f{T}_i^a,T_i^c\le \theta +f{T}_i^d$$

(44)

The disadvantage of the CLS lies in the challenging computations to solve the minimization problem. Therefore, two additional least squares estimators, referred to as the feasibility checked least squares (FLS) estimator and the Paxson-based estimator, respectively, were proposed in [39] to reduce the computational complexity. Experimental results showed that the FLS estimator and the Paxson-based estimator perform comparably and sometimes better than the CLS estimator, while exhibiting the advantage of much less computational complexity.

6. Fully-Distributed Clock Synchronization Algorithms

The centralized signal processing techniques can only help address the problem of pairwise synchronization. As discussed in Section 1, the network-wide synchronization requires a specified network structure to extend the pairwise synchronization approach to all of the sensor nodes. This increases the overhead of the communication and makes the network vulnerable to node failures. More recent works on clock synchronization mainly concentrated on applying decentralized signal processing methods (e.g., distributed estimation techniques) to develop clock synchronization algorithms in a fully-distributed manner.

6.1. System Model

In a fully-distributed clock synchronization algorithm, the synchronization is not performed pairwisely, but simultaneously among all of the sensor nodes. Therefore, instead of estimating the clock offset difference and the relative clock skew between two nodes, we aim to find the clock offset and skew for each individual sensor node. A two-way message exchange between nodes i and j in the n-th round of their message exchange is modeled in Figure 4. Following the same message exchange procedure described in Section 2.2, we can write:

$$\frac{1}{f_j}[c_j\left(t_n^2\right)-{\theta }_j]=\frac{1}{f_i}[c_i\left(t_n^1\right)-{\theta }_i]+d_{ij}+X_{ij}$$

(45)

$$\frac{1}{f_j}[c_j\left(t_n^3\right)-{\theta }_j]=\frac{1}{f_i}[c_i\left(t_n^4\right)-{\theta }_i]-d_{ji}-X_{ji}$$

(46)

where $d_{ij}$ and $d_{ji}$ denote the fixed delays from node i to node j and from node j to node i, respectively, and $X_{ij},X_{ji}$ stand for the random delays. If only the effect of clock offset is considered, i.e., $f_i=f_j=1$, and the network topology is assumed stationary, i.e., $d_{ij}=d_{ji}=d$, the system model in Equations (45) and (46) reduces to the more simplified model:

$$c_j\left(t_n^2\right)-{\theta }_j=c_i\left(t_n^1\right)-{\theta }_i+d+X_{ij}$$

(47)

$$c_j\left(t_n^3\right)-{\theta }_j=c_i\left(t_n^4\right)-{\theta }_i-d-X_{ji}$$

(48)

Figure 4. Two-way message exchange between nodes i and j.

Figure 4. Two-way message exchange between nodes i and j.

6.2. Fully-Distributed Clock Synchronization Algorithms under Gaussian Delays

The Gaussian assumption for the random delay component of WSNs is justifiable due to the CLT and many experimental results (see, e.g., [22]). The state-of-the-art of fully-distributed clock synchronization algorithms under Gaussian random delays are discussed in the following sub-section.

6.2.1. Belief-Based Synchronization Algorithms

Existing fully-distributed algorithms in [22,46,47] are based on the belief propagation method in the Bayesian framework. Under the assumption of Gaussian random delays, [46] proposed a belief-based algorithm to estimate the clock offset in a distributed manner. Specifically, the system model is modified by adding Equation (47)–(48), and it leads to:

$$T_{\{i,j\},n}\stackrel{\mathrm{\Delta }}{=}{c}_j\left(t_n^2\right)+c_j\left(t_n^3\right)-c_i\left(t_n^1\right)-c_i\left(t_n^4\right)=2{\theta }_j-2{\theta }_i+Z_n$$

(49)

where $Z_n\stackrel{\mathrm{\Delta }}{=}{X}_{ij}-X_{ji}$. Stacking all of the observations in a vector as ${\mathbf{T}}_{i,j}=\left[T_{\{i,j\},1},\cdots ,T_{\{i,j\},N}\right]$ leads to the likelihood function denoted as $p\left({\mathbf{T}}_{i,j}\right|{\theta }_i,{\theta }_j)$. In the Bayesian scenario, the clock offset of each sensor node, denoted as ${\theta }_i$, is regarded as a random variable with prior distribution $p\left({\theta }_i\right)$. The estimate is obtained by maximizing the posterior distribution of the clock offset.

The posterior distribution of ${\theta }_i$ for a network with only two nodes i and j is expressed as:

$$g_i\left({\theta }_i\right)=\int p({\theta }_i,{\theta }_j|{\mathbf{T}}_{i,j})d{\theta }_j\propto \int p\left({\mathbf{T}}_{i,j}\right|{\theta }_i,{\theta }_j)p\left({\theta }_i\right)p\left({\theta }_j\right)d{\theta }_j$$

Extending the above idea to a WSN with M sensor nodes, the posterior distribution of clock offset ${\theta }_i$ is expressed as:

$$g_i\left({\theta }_i\right)=\underset{{\theta }_1,\cdots ,{\theta }_{i-1},{\theta }_{i+1},\cdots ,{\theta }_M}{\int \cdots \int }p({\theta }_1,\cdots ,{\theta }_M|{\left\{{\mathbf{T}}_{ij}\right\}}_{i=1,\cdots ,M,j\in {\beta }_i})d{\theta }_1\cdots d{\theta }_{i-1}d{\theta }_{i+1}d{\theta }_M$$

(50)

where ${\beta }_i$ stands for the indices of neighboring sensors of node i. It can be seen that the joint distribution depends on interactions among all of the variables, and the integration is almost impossible to calculate. Therefore, belief propagation (BP) is employed to compute the posterior distribution without fully integrating Equation (50).

BP takes advantage of the graphical model structure. Factor graphs represent one of the most widely used graphical models and have been adopted in [46]. An example of a factor graph for two nodes i and j is depicted in Figure 5. The two sensor nodes i and j are represented in terms of the variable nodes (circles), and they are connected to a square factor node ${\gamma }_{ij}$, which stands for the likelihood function $p\left({\mathbf{T}}_{ij}\right|{\theta }_i,{\theta }_j)$. On the other hand, nodes i and j are also connected to factor nodes ${\gamma }_i$ and ${\gamma }_j$, which represent the prior distributions of ${\theta }_i$ and ${\theta }_j$, respectively. During each iteration, i.e., for the iteration l, the BP algorithm in [46] is carried out via the following three steps:

(1)

Each variable node ${\theta }_i$ transmits its current belief $b_i^{\left(l\right)}\left({\theta }_i\right)$ to all of its neighboring factornodes $\{{\gamma }_{ij},j\in {\beta }_i\}$.

(2)

Acting like an intermediate node, the message from a factor node ${\gamma }_{ij}$ to a variable node ${\theta }_i$ is calculated based on the belief received from ${\theta }_j$:

$$m_{{\gamma }_{i,j}\to {\theta }_i}^{\left(l\right)}\left({\theta }_i\right)=\int p\left({\mathbf{T}}_{i,j}\right|{\theta }_i,{\theta }_j)b_j^{\left(l\right)}\left({\theta }_j\right)d{\theta }_j$$

(51)

(3)

After variable node ${\theta }_i$ receives all of the messages from its neighboring factor nodes, i.e., ${\{m_{{\gamma }_{i,j}\to {\theta }_i}^{\left(l\right)}\left({\theta }_i\right)\}}_{j\in {\beta }_i}$, it updates its belief $b_i^{(l+1)}\left({\theta }_i\right)$ as follows:

$$b_i^{(l+1)}\left({\theta }_i\right)=p\left({\theta }_i\right)\prod _{j\in {\beta }_i}{m}_{{\gamma }_{i,j}\to {\theta }_i}^{\left(l\right)}\left({\theta }_i\right)$$

(52)

Figure 5. The factor graph of nodes i and j.

Figure 5. The factor graph of nodes i and j.

If no prior information is given about $p\left({\theta }_i\right)$, we set $p\left({\theta }_i\right)$ to a Gaussian PDF with zero mean and very large variance. Furthermore, the belief of each node is initially set as its prior distribution, i.e., $b_i^{\left(0\right)}\left({\theta }_i\right)=p\left({\theta }_i\right)$. Beliefs and messages are then iteratively updated at variable nodes and factor nodes, respectively. After convergence, the belief at each node reaches the posterior distribution exactly when the network topology is loop free and approximately when the network topology presents loops [48]. Thus, the maximum a posteriori probability (MAP) estimate is obtained by maximizing the converged belief at each node. Since each node runs the same algorithm only with its neighbors, the estimates are achieved in a fully-distributed manner.

However, only the problem of clock offset estimation is investigated in [46]. Moreover, the algorithm is performed in a synchronous way, i.e., every node updates its belief only after receiving messages from all of its neighboring factor nodes. Due to the random packet dropouts, the synchronous algorithm may converge slowly. To account for these aspects, Du et al. [47] generalized the belief-based algorithm to jointly estimate the clock offset and skew under Gaussian random delays using both synchronous and asynchronous algorithms. In the asynchronous algorithm, some nodes are allowed to update their beliefs more frequently as long as they receive messages from their neighbors within a preset time period.

Following the two-way message exchange model in Equations (45) and (46), the authors in [47] eliminated the fixed delay d by adding these two equations together. It turns out that:

$${\mathbf{A}}_{j,i}{\beta }_j+{\mathbf{A}}_{i,j}{\beta }_i=z_{j,i}$$

(53)

where ${\mathbf{A}}_{j,i}$ and ${\mathbf{A}}_{i,j}$ are N-by-two matrices with the n-th row representing $[c_j\left(t_n^2\right)+c_j\left(t_n^3\right),-2]$ and $-[c_i\left(t_n^1\right)+c_i\left(t_n^4\right),-2]$, respectively, ${\beta }_j\stackrel{\mathrm{\Delta }}{=}{[1/f_j,{\theta }_j/f_j]}^T$, ${\beta }_i\stackrel{\mathrm{\Delta }}{=}{[1/f_i,{\theta }_i/f_i]}^T$ and $z_{j,i}={[X_{j,1}-Y_{j,1},\cdots ,X_{j,N}-Y_{j,N}]}^T$. Thus, the likelihood function can be obtained from Equation (53), and it is denoted by $p({\mathbf{A}}_{i,j},{\mathbf{A}}_{j,i}|{\beta }_i,{\beta }_j)$.

The message passing procedure adopted in [47] is slightly different from the one in [46]. In Step 1, instead of broadcasting the belief to all of its neighboring factor nodes, the message transmitted from variable node ${\theta }_i$ to factor node ${\gamma }_{ij}$ is evaluated via:

$$m_{{\theta }_i\to {\gamma }_{i,j}}^{\left(l\right)}\left({\beta }_i\right)=\prod _{k\in {\beta }_i\setminus j}{m}_{{\gamma }_{i,k}\to {\theta }_i}^{(l-1)}\left({\beta }_i\right)$$

(54)

which is simply the product of the incoming messages on the other links. Thus, in Step 2, the message from a factor node ${\gamma }_{ij}$ to a variable node ${\theta }_i$ is updated via:

$$m_{{\gamma }_{i,j}\to {\theta }_i}^{\left(l\right)}\left({\beta }_i\right)=\int p({\mathbf{A}}_{i,j},{\mathbf{A}}_{j,i}|{\beta }_i,{\beta }_j)m_{{\theta }_j\to {\gamma }_{i,j}}^{\left(l\right)}\left({\beta }_j\right)d{\beta }_j$$

(55)

and the belief is further updated as:

$$b_i^{(l+1)}\left({\beta }_i\right)=\prod _{j\in {\beta }_i}{m}_{{\gamma }_{i,j}\to {\theta }_i}^{\left(l\right)}\left({\beta }_i\right)$$

(56)

It is illustrated analytically in [47] that the asynchronous algorithm converges regardless of the network topology, and the MSE of the estimation parameters achieves the centralized CRLB asymptotically. Simulation results further show that the asynchronous algorithm converges faster than the synchronous one.

In parallel to the work in [47], Etzlinger et al. [22] proposed a BP algorithm and a mean field (MF) algorithm and adopted a Bayesian model with Gaussian random delays. The MAP estimates of clock offset and skew are derived using some simplifications of the measurement likelihoods and prior distributions. The message update rule in the BP algorithm is analogous to the one in [47] (see, e.g., Equations (54) and (55)). As concerns the implementation of the MF algorithm, each variable node ${\theta }_i$ broadcasts its current belief $b_i^{\left(l\right)}\left({\theta }_i\right)$ to all of its neighboring factor nodes, and the same step as in [46] is followed. However, in the second step, the message transmitted from a variable node ${\theta }_i$ to a factor node ${\gamma }_{i,j}$ is alternatively evaluated as:

$$m_{{\gamma }_{i,j}\to {\theta }_i}^{\left(l\right)}\left({\theta }_i\right)=\text{exp}\left(\int log\left(p\left(T_{i,j}\right|{\theta }_i,{\theta }_j)\right)b_j^{\left(l\right)}\left({\theta }_j\right)d{\theta }_j\right)$$

(57)

Numerical results corroborate the attractive performances exhibited by both BP and MF algorithms.

6.2.2. Consensus-Based Synchronization Algorithms

The fully-distributed algorithms reported in [49,50,51] rely on the average consensus principle. However, the message delays are not considered in these algorithms, and their performance degenerates significantly when message delays exist [46]. Other consensus-based distributed synchronization algorithms, such as [52,53], typically suffer from slow convergence speed and require a high number of data packages [22]. To overcome this issue, Berger et al. [54] recently presented a fully-distributed low-complexity consensus-based synchronization algorithm for embedded WSNs. Another fully-distributed clock synchronization method based on a cascade of two consensus algorithms was proposed by Schenato et al. [55], which exhibits the advantage of being asynchronous and being robust to packet losses, node failures and communication topology changes. The details of these consensus-based algorithms are omitted herein, since they are not based on the statistical framework described in Section 6.1. Xiong et al. [56] generalized the consensus-based algorithm to estimate the clock offset by taking into account both fixed delays, as well as random Gaussian delays. In each iteration of the generalized consensus-based algorithm, each node processes and decodes the timestamps from its neighbors and then updates its local clock time using a weighted average of the time differences with its neighbor nodes. Such a strategy naturally leads to the introduction of a special network topology referred to as the time delay balanced network in which average timing consensus is achieved in the presence of Gaussian random delays.

6.3. Fully-Distributed Clock Synchronization Algorithms under Exponential Delays

Compared to the fully-distributed clock synchronization algorithms that assume Gaussian network delays, fewer works have been proposed to account for exponential network delays in a fully-distributed manner. Considering only the estimation of clock offset, Zennaro et al. [57] pioneered a fully-distributed algorithm under exponential delays based on a factor-graph representation of the network and a max-product message update algorithm. From the two-way message exchange model in Equations (47) and (48), since the MLE for the clock offset difference ${\theta }_j-{\theta }_i$ is given by $S_{ij}=\left[U_{ij,\left(1\right)}-V_{ij,\left(1\right)}\right]/2$, where $U_{ij,\left(1\right)}$ and $V_{ij,\left(1\right)}$ denote the first order statistics of $U_{ij,n}\stackrel{\mathrm{\Delta }}{=}{c}_j\left(t_n^2\right)-c_i\left(t_n^1\right)$ and $V_{ij,n}\stackrel{\mathrm{\Delta }}{=}{c}_i\left(t_n^4\right)-c_j\left(t_n^3\right)$, respectively, the modified system model is expressed as:

$$S_{ij}={\theta }_j-{\theta }_i+Z_{ij}$$

(58)

where $Z_{ij}\stackrel{\mathrm{\Delta }}{=}\left(X_{ij,\left(1\right)}-Y_{ij,\left(1\right)}\right)/2$ is a Laplace random variable, since $X_{ij}$ and $Y_{ij}$ are both assumed to be exponentially distributed. Thus, the likelihood function $p\left(S_{ij}\right|{\theta }_i,{\theta }_j)$ is formulated based on the model in Equation (58). The factor graph representation of the network is identical to the one employed in [46]. In terms of the message update rule, the message transmitted from variable node ${\theta }_i$ to factor node ${\gamma }_{ij}$ is given by:

$$m_{{\theta }_i\to {\gamma }_{i,j}}^{\left(l\right)}\left({\theta }_i\right)=p\left({\theta }_i\right)\prod _{k\in {\beta }_i\setminus j}{m}_{{\gamma }_{i,k}\to {\theta }_i}^{(l-1)}\left({\theta }_i\right)$$

(59)

Moreover, the message from a factor node ${\gamma }_{i,j}$ to a variable node ${\theta }_i$ is updated by the “max” operator:

$$m_{{\gamma }_{i,j}\to {\theta }_i}\left({\theta }_i\right)=\underset{{\theta }_j}{max}[m_{{\theta }_j\to {\gamma }_{i,j}}\left({\theta }_j\right)p\left(S_{ij}\right|{\theta }_i,{\theta }_j)]$$

(60)

and the belief update is expressed as:

$$b_i^{\left(l\right)}=p_i\left({\theta }_i\right)\prod _{j\in {\beta }_i}{m}_{{\gamma }_{i,j}\to {\theta }_i}\left({\theta }_i\right)$$

(61)

Under exponential random delays, [58] proposed a fully-distributed synchronization algorithm for joint estimation of clock offset and skew. The joint maximum likelihood estimation of the clock offset and skew is first formulated as a linear programming (LP) problem and solved in a centralized way. With the help of auxiliary replica variables, the original LP problem is re-formulated as an equivalent optimization problem with the structure more amenable for decomposition, which is further decoupled into a series of subtasks. By applying the classic alternating direction method of multipliers (ADMM), these subtasks are solved in a distributed way while still guaranteeing the convergence to the centralized solution.

7. Conclusions and Open Problems

Wireless sensor networks can be applied to a variety of applications, and most of these applications require synchronization among the sensor nodes. Based on the two-way message exchange system model described in Section 2.2, this paper surveyed the most representative pairwise and fully-distributed clock synchronization algorithms from a statistical signal processing viewpoint. The interested reader is referred to [59,60,61,62,63] for references based on other system models. Specifically, similar to Equation (2), [59] adopted a two-way message exchange mechanism where no random delay is considered and the fixed portion of delays is different for the uplink and downlink (i.e., replace d in Equation (2) by $d_{\text{AB}}$ and $d_{\text{BA}}$). The main result is that while estimating the clock skew between two nodes is possible, it is impossible to determine the clock offset for pairwise synchronization, unless the delays in two-way message communication are symmetric, i.e., $d_{\text{AB}}=d_{\text{BA}}$. The same result was later extended in [60] for the network case. Recently, timestamp-free synchronization algorithms were reported in [61,62] to avoid the overhead brought by timestamp exchanges. The approach employed in [61] is based on conveying implicit timing information in the physical layer via the timing responses from the receiver to the transmitter. The work in [62] avoids timestamp exchanges using the round-trip-time data, which can be measured by a time-to-digital converter in the master node. The clock parameters, such as the offset and skew, as well as the unknown range between the master and slave nodes were experimentally estimated and verified with an accuracy in the order of a nano-second. Without considering the random delays, a fully-distributed network-wide synchronization algorithm was presented in [63] by assuming the constraint that the relative clock offsets in any network loop sum to zero. Recently, Jeske [64] developed a statistical process control (SPC) algorithm that can be used in the context of the two-way message exchange mechanism to detect translations in the delay distribution.

In reflecting on the research problems which have been addressed, we have observed several open research topics that need further investigation. On the one hand, the algorithms presented in this article all assume a line-of-sight transmission. The fixed portion of delays d herein is actually related to the propagation delay between the transmitter and the receiver nodes. It may occur that some obstacles exist in the signal path, and they might reflect or simply absorb the signal. In this case, the received signal strength and the propagation delay may be significantly affected, and thus, the mathematical system model proposed in this article might not be plausible. To our best knowledge, the clock synchronization problem under non-line-of-sight transmissions is still open. In addition, even though many references that exploit the statistical system model from Section 2.2 have been reported in the literature, more convincing experimental results based on large-scale WSN testbeds are necessary for a more accurate clock synchronization system model. Other possible future research topics lie in the investigation of the convergence rate and computational complexity of the proposed distributed synchronization algorithms. Due to the limited energy of sensors in WSNs, designing new distributed synchronization schemes with reduced energy consumption, low implementation complexity and a fast convergence rate represents another fruitful research direction.