A Novel Carrier Loop Algorithm Based on Maximum Likelihood Estimation (MLE) and Kalman Filter (KF) for Weak TC-OFDM Signals

Abstract

Digital broadcasting signals represent a promising positioning signal for indoors applications. A novel positioning technology named Time & Code Division-Orthogonal Frequency Division Multiplexing (TC-OFDM) is mainly discussed in this paper, which is based on China mobile multimedia broadcasting (CMMB). Signal strength is an important factor that affects the carrier loop performance of the TC-OFDM receiver. In the case of weak TC-OFDM signals, the current carrier loop algorithm has large residual carrier errors, which limit the tracking sensitivity of the existing carrier loop in complex indoor environments. This paper proposes a novel carrier loop algorithm based on Maximum Likelihood Estimation (MLE) and Kalman Filter (KF) to solve the above problem. The discriminator of the current carrier loop is replaced by the MLE discriminator function in the proposed algorithm. The Levenberg-Marquardt (LM) algorithm is utilized to obtain the MLE cost function consisting of signal amplitude, residual carrier frequency and carrier phase, and the MLE discriminator function is derived from the corresponding MLE cost function. The KF is used to smooth the MLE discriminator function results, which takes the carrier phase estimation, the angular frequency estimation and the angular frequency rate as the state vector. Theoretical analysis and simulation results show that the proposed algorithm can improve the tracking sensitivity of the TC-OFDM receiver by taking full advantage of the characteristics of the carrier loop parameters. Compared with the current carrier loop algorithms, the tracking sensitivity is effectively improved by 2–4 dB, and the better performance of the proposed algorithm is verified in the real environment.

1. Introduction

With the development of mobile Internet and mobile smart devices, location-based services have become the focus in many wireless network applications. In open outdoor areas, Global Navigation Satellite Systems (GNSS) can provide precise location information for outdoor activities, however, in buildings, urban canyons, and forested areas, the effectiveness of positioning using GNSS is limited, making it impossible to provide high-precision location-based services. Typical studies on the above problems in such harsh environments can be found in [1,2,3,4,5]. A general framework to characterize the localization accuracy by unifying localization information of wideband wireless networks is proposed in [1]. Reference [2] established the fundamental limits of wideband cooperative location-aware networks. Centimeter accuracy indoor localization in an Assisted Living (AL) system is achieved for a 5G systems using millimeter-wave signals, which can be found in [3]. Reference [4] proposed a range information model which is a function of wireless environment, signal features and energy detection techniques. Soft range information-based localization can be found in [5] for high accuracy network localization. Meanwhile, precise localization is of great importance in many applications such as vehicular networks [6]. A state of the art vehicular communication system for the cooperative awareness of connected vehicles is proposed in [7].

Studies show that people spend more than 80% of their time in indoors environments [8]. With the increasing number of tall buildings, indoor location-based services have emerging applications in commercial applications and public safety. As the main positioning technology, GNSS has many significant advantages outdoors, such as a large coverage area, high positioning accuracy, and superior navigation performance [9]. However, buildings obstruct, reflect, and diffract the GNSS signals, the positioning performance is limited in indoor environments and cities. Recently, terrestrial radio positioning systems and their enhancements to the GNSS have drawn increasing attention, and the digital broadcasting signal is a promising positioning signal for indoors uses [10]. As for DVB-T/H signals, a different low-complexity solution to perform the coupling channel estimation in an on-channel repeater can be found in [11,12,13]. This paper mainly studies a novel terrestrial radio positioning system called Time & Code Division-Orthogonal Frequency Division Multiplexing (TC-OFDM), which is based on the China Mobile Multimedia Broadcasting (CMMB) system. The TC-OFDM system multiplexes the CMMB signal and pseudorange noise (PRN) codes in the same frequency band, and the positioning performance can be achieved by adding some simple modifications to the deployed CMMB facilities. The positioning part of the TC-OFDM system is a direct-sequence spread spectrum code division multiple access (DSSS-CDMA) system employing binary phase shift keying (BPSK) modulation, and [10] describes the TC-OFDM system in detail. Compared with the GNSS, the TC-OFDM system has many potential advantages: the TC-OFDM signal transmission power is stronger, the frequency band is the U band. The TC-OFDM receiver demodulates PRN codes for high accuracy positioning, which can overcome the positioning limitations caused by the CMMB single network coverage.

Weak signal strength is the main factor that limits the performance of the receiver tracking loop to improve tracking sensitivity. Meanwhile, the indoor environment is more complex than outdoors and the signal strength is significantly attenuated, which restricts the tracking performance of the existing receivers for weak TC-OFDM signals. The main reason is that the accuracy of the residual carrier estimation by the existing carrier loop algorithm is not ideal under weak signal conditions. Since the carrier loop is the weakest link in the receiver tracking loop, the superior carrier loop performance can improve the tracking sensitivity of the receiver for weak TC-OFDM signals to a certain degree [14]. The carrier loop is usually divided into a frequency-locked loop (FLL) and phase-locked loop (PLL) [15,16]. The FLL has good dynamic performance, but the carrier phase measurement accuracy is lower due to the wide noise bandwidth [17]. The PLL tracks the received signal more closely and has the high carrier phase measurement accuracy, but it has poor tolerance of dynamic stress [18]. Due to the complexity of the indoor environment, the received signal strength is greatly attenuated. Therefore, the existing carrier loop performance needs to be improved for better performance in indoor weak signal environment. Currently, the methods to improve the carrier loop performance are divided into three categories: the KF-based PLL [19], the algorithms based on the Maximum likelihood estimation (MLE) algorithm [20], and the FLL-assisted PLL [21]. The KF-based PLL includes Extended Kalman Filter (EKF) and Unscented Kalman Filter (UKF) algorithms [22,23]. The observation model in the carrier loop is nonlinear, which means the observed signal vector and the estimated state vector are nonlinear in the carrier loop. The EKF and UKF are utilized to expand the nonlinear observation functions using Taylor series and keep the linear part of the function while ignoring the higher order nonlinear part. However, the EKF and UKF based on linear minimum mean square error estimation criterion represent a sub-optimal carrier loop scheme, which still produces a large parameter estimation error for weak received signals. Methods based on FLL-assisted PLL can ensure the dynamic of the carrier loop but the estimation accuracy is limited in a weak signal environment [24]. Reference [25] uses the MLE to adjust the Doppler frequency in a high dynamics receiver carrier loop, but it does not make a detailed analysis for weak received signals. Iterative and non-iterative MLE are proposed to estimate the signal Doppler frequency and code phase in [26,27]. These two algorithms need to deal with real-time IF data, which means a heavy calculation burden.

This paper proposes a novel carrier loop algorithm based on MLE and KF for weak TC-OFDM signals. The main idea is to combine the MLE with KF to effectively estimate the carrier loop parameters and smooth the estimation results, so as to obtain an accurate estimation of the residual carrier for weak TC-OFDM signals and improve the receiver tracking sensitivity indoors. With the novel algorithm, the MLE discriminator function replaces the current carrier loop discriminator. The Levenberg-Marquardt (LM) algorithm is utilized to obtain the MLE cost function consisting of signal amplitude, residual carrier frequency and carrier phase, and the MLE discriminator function is derived from the corresponding MLE cost function. The KF is used to smooth the MLE discriminator function results, which takes the carrier phase estimation, the angular frequency estimation and the angular frequency rate as the state vector.

The rest of this paper is organized as follows: Section 2 presents the signal model. The proposed carrier loop algorithm is introduced in Section 3. Section 4 presents the simulation and real data test results of the comparison between different carrier loop algorithms. Finally, Section 5 concludes the work.

2. Signal Model

The CMMB signal and pseudorange noise (PRN) codes are multiplexed in the same frequency band. Figure 1 shows the structure of the TC-OFDM signal. The duration of each frame of the CMMB signal is 1 s, and each frame consists of 40 time slots. The duration of each time slot is 25 ms. The CMMB system uses orthogonal frequency division multiplexing (OFDM) to transmit the broadcasting signal. Two kinds of pseudo codes are superimposed on the CMMB signals to provide high accuracy positioning services, which are named short codes and long codes, respectively. The beginning of the frame has the TxID and two synchronization signals. The TxID is empty in the practice CMMB signal, so the short codes are superimposed on the TxID to distinguish different base stations by using different pseudo codes signals. Thanks to the fact the TxID is empty, the short codes and the CMMB signals have the same transmission power. In order not to affect the normal communication of the CMMB signal, the power of long codes is lower than the CMMB signal by 20 dB [10]. In the positioning, the receiver uses short codes to achieve acquisition and long codes to achieve stable tracking.

The TC-OFDM signal of the nth time slot can be expressed as:

$$s_k^{(i)}(t)=\{\begin{array}{ll}{s}_{CMMB}(t)+c_{SC}^{(i)}(t)& (k-1)T_F\le t<(k-1)T_F+T_{SC}\\ s_{CMMB}(t)+\alpha c_{LC}^{(i)}(t)& (k-1)T_F+T_{SC}\le t\le k{T}_F\\ 0& others\end{array}$$

(1)

where the S_CMMB(t) is the CMMB signal, C_SC(t) and C_LC(t) is short codes and long codes respectively. The superscript i represents the base station number. T_F is the time length of the time slot and T_SC is the time length of the short codes. The term α is the signal attenuation factor.

In order to transmit the positioning-assisted information, the PRN codes sent by the base station are modulated with the positioning-assisted information, and each slot modulates one bit positioning-assisted information. The TC-OFDM signal of ith base station can be written as:

$$s^{(i)}(t)={\displaystyle \sum _{k=-\infty }^{\infty }{d}^{(i)}(t)}{s}_k^{(i)}(t)$$

(2)

where d(t) denotes the positioning-assisted information. The signal transmitted by the ith base station is:

$$S^{(i)}(t)=s^{(i)}(t)\cos (2\pi f_{c}t+{\phi }_{0,i}(t))$$

(3)

where f_c is the carrier frequency, φ_0,i(t) is the initial phase.

The signal from the ith base station is received by the receiver radio frequency (RF) antenna. The output signal of the RF antenna can be written as:

$$r(t)={\displaystyle \sum _{i=1}^N{A}^{(i)}}{s}^{(i)}(t-{\tau }_i)\cos (2\pi (f_c+f_{d,i})t+{\phi }_{0,i}(t))+\omega (t)$$

(4)

where N denotes the signal received from N different base stations. A⁽ⁱ⁾ is the signal amplitude, τ_i is the incoming signal delay, f_c is the carrier frequency, f_d,i is the incoming Doppler shift, φ_0,i(t) is the initial phase, ω(t) is the additive Gaussian white noise (AWGN) with zero mean and variance σ_n².

After the receiver RF front-end, down-conversion, low-pass filter and analog-digital conversion module (ADC), the digital intermediate frequency (IF) signal is expressed as:

$$r_{IF}{}^{(i)}(n{T}_s)=A_{IF}{s}^{(i)}(n{T}_s-{\tau }_i)e^{j2\pi (f_{IF}+f_{d,i})n{T}_s+{\phi }_{0,i}}+\omega (n)$$

(5)

where T_s is the sampling duration, A_IF is the IF signal amplitude, τ_i is the incoming signal delay, f_IF is the IF frequency, f_d,i is the Doppler shift, φ_0,i is the initial carrier phase, ω(n) is the additive Gaussian white noise (AWGN) with zero mean and variance σ_n².

The IF signal is sent to the baseband processor for acquisition [10], tracking and data demodulation. A rough estimation of carrier frequency and code phase of the received signal is obtained from the acquisition process, which initialize the related parameters of the tracking channel. Then the estimation of these two signal parameters is refined step by step through the tracking loop. The numerically controlled oscillator (NCO) in the tracking loop produces mutually orthogonal sine and cosine signals, which can be created as:

$$\{\begin{array}{c}{\mu }_{os}(t)=\sin (2\pi f_{NCO}t+{\phi }_{NCO})\\ {\mu }_{oc}(t)=\cos (2\pi f_{NCO}t+{\phi }_{NCO})\end{array}$$

(6)

where $f_{NCO}$ and ${\phi }_{NCO}$ is the carrier NCO frequency and initial phase respectively.

In this research, the TCXO is used in the TC-OFDM receiver, and the accuracy of the TCXO is 0.5 ppm. After many real tests, we find that the phase noise of the receiver clock is smaller than white Gaussian noise in the research of the TC-OFDM meter-level positioning accuracy. Also, we can use the TCXO phase noise estimation algorithm to predict the actual phase noise. However, adding the TCXO phase estimation algorithm has less influence on the proposed algorithm in this research. Therefore, the effect of phase noise can be ignored in this research. After carrier stripped, the coherent integration result of I and Q channel signals by integration and dumping module can be created as:

$$\{\begin{array}{c}Corr(n)=A_{mp}s(n)\sin c(\Delta f{T}_{coh})e^{(j(2\pi \Delta fn{T}_{coh}+\Delta \phi )}+\omega (n)\\ A_{mp}=A_{IF}(n)T_{coh},\Delta f=f_d-f_{NCO},\Delta \phi ={\phi }_{IF}-{\phi }_{NCO}\end{array}$$

(7)

where Corr(n) represents the nth coherent integration result, T_coh is the coherent integration duration, s(n) denotes the positioning-assisted information, A_mp, Δf and Δφ represent the signal amplitude, carrier and phase residual respectively. In order to avoid bit transition, the coherent integration duration used generally do not exceed one time slot duration. We assume that in the coherent integration duration, s(n) can be regarded as a fixed value.

For the convenience of analysis, we assume that the local code is strictly aligned with the received signal. When in the coherent integration duration, the parameters A_mp, Δf and Δφ have minor changes which can be regarded as unknown nonrandom parameters. In the tracking process, the term of sinc(ΔfT_coh) ≈ 1 when Δf is small enough. Then Equation (8) can be simplified as:

$$Corr(n)\approx A_{mp}s(n)e^{(j(2\pi \Delta fn{T}_{coh}+\Delta \phi )}+\omega (n)$$

(8)

The conventional carrier loop structure is shown in Figure 2. The coherent integration results of Equation (8) is sent into the carrier discriminator to estimate carrier frequency. Then the Carrier NCO is adjusted in real time based on the carrier frequency filtering results from the loop filter.

3. MLE Parameter Estimation Model

In the case of an unknown prior distribution of estimated parameters, MLE provides the parameter estimation with the smallest error variance when the parameters to be estimated is symmetric and unimodal distributions. In terms of the linear discrete-time system, linear KF is the optimal filter [28]. Therefore, the MLE combined with KF can make an accurate estimation of the parameters in the carrier loop, which can improve the tracking loop performance. The MLE is used to nonlinearly estimate the carrier loop parameters of interest, and then KF is applied to smooth the estimated parameters from MLE. Therefore, the proposed algorithm of combining MLE with KF is the optimal solution for the TC-OFDM carrier frequency accurately tracking in the weak TC-OFDM signal environment.

3.1. The Principle of MLE

MLE is often used to estimate unknown non-random parameters. MLE is defined as the estimator of θ value that maximizes the likelihood function. The estimated value is denoted as θ_MLE(x). We take the MLE for a single-parameter θ as an example. For an unknown non-random estimation θ, the probability density function p(x|θ) of the observation vector (denotes as x) is called the likelihood function. The basic principle of MLE can be explained as follows: for a selected θ, MLE consider the probability p(x|θ)dx that x falls in a small area, and take the corresponding θ whose p(x|θ)dx is the largest as the estimated θ_MLE(x). As shown in Figure 3, the likelihood function is obtained after x = x₀, then we can draw a curve between the likelihood function and the estimated value of θ. The value of p(x|θ)dx for each θ means the probability that x falls within dx, and dx centered on x₀ in the observation space R for θ. When x = x₀, it can be seen that θ = θ₁ is unreasonable, so we choose θ = θ₂ as the estimation, and the θ that maximizes p(x = x₀|θ) is selected as the estimated value θ_MLE(x) within a range allowed by the estimated θ.

3.2. MLE Cost Function

Given a set of observations x, the MLE of θ is the θ_MLE(x) at which the probability density function is maximized:

$${\theta }_{MLE}(x)=\arg \underset{\theta }{\max }p(x|\theta )$$

(9)

According to the above analysis, when the likelihood function p(x|θ) is known, θ_MLE(x) can be obtained by the following equation and the logarithmic form is given in Equation (11):

$${\frac{\partial p(x|\theta )}{\partial \theta }|}_{\theta ={\theta }_{MLE}}=0$$

(10)

$${\frac{\partial \ln p(x|\theta )}{\partial \theta }|}_{\theta ={\theta }_{MLE}}=0$$

(11)

Since MLE is considered a valid parameter estimation algorithm in the tracking loop of the TC-OFDM receiver, we use MLE to effectively estimate the residual carrier frequency for weak TC-OFDM signals indoors. Meanwhile, the MLE produces the minimum variance of estimation error, when the statistical distribution of the estimated signal parameters in an uncertain interval is unknown [29]. When it comes to multi-parameter problems, the MLE of signal parameters is the estimation of signal parameters when the joint conditional probability density of a set of signal observations gets maximized. Since the coherent integration time in the algorithm is 1.6 ms which is shorter than the loop update time, we assume that the carrier is correctly tracked within such a short coherent integration time. Therefore, the parameters to be estimated can be regarded as unknown constants over the interval. Meanwhile, N is defined as the number of consecutive coherent integration results in this paper. The coherent integration results are adopted as observation sample parameters of MLE, which contains the signal amplitude, residual carrier and phase value in Equation (8). The value of N is selected to ensure that the parameters to be estimated remain constant over the entire interval, and N is closely related to the dynamic bandwidth of the receiver. The TC-OFDM system is for indoor positioning, which is mainly under low dynamic cases. In the low dynamic case, signal amplitude is the main change parameter. We use the MLE to estimate the signal amplitude, and other parameters (including residual carrier and phase) are little changed under such the low dynamic case. Also, the simulation and real test are used to determine the value of N in this research. Therefore, in order to obtain the MLE of the unknown A_mp, Δf and Δφ in Equation (8), the joint probability density function of N consecutive coherent integration results is obtained according to [30]:

$$p(Cor{r}_N|A_{mp},\Delta f,\Delta \phi )=\frac{1}{(2\pi {\sigma }^2{)}^N}{e}^{(-\frac{1}{2{\sigma }^2}(Cor{r}_N-{\stackrel{\wedge }{Corr}}_N)W{(Cor{r}_N-{\stackrel{\wedge }{Corr}}_N)}^H)}$$

(12)

where Corr_N = [Corr(0), Corr (1), ……, Corr (N − 1)] represents N consecutive coherent integration results, ${\stackrel{\wedge }{Corr}}_N$ is the estimated value of $Cor{r}_N$. W is the diagonal matrix which stands for the weight factor of Corr(n), superscript H denotes the matrix transpose and conjugate operations.

The MLE minimum variance unbiased estimation can be achieved by obtaining the maximum value of (10) to get the values A_mp, Δf and Δφ. The diagonal elements of the diagonal matrix W are set as 1 to obtain the log-likelihood cost function of Equation (12) as in [31,32]:

$$\begin{array}{l}\Lambda (\mu |Cor{r}_N)=-\frac{1}{2{\sigma }^2}(Cor{r}_N-{\stackrel{\wedge }{Corr}}_N){(Cor{r}_N-{\stackrel{\wedge }{Corr}}_N)}^H-N\ln (2\pi {\sigma }^2)\\ =-\frac{1}{2{\sigma }^2}{\left|Cor{r}_N-{\stackrel{\wedge }{Corr}}_N\right|}^2-N\ln (2\pi {\sigma }^2)\\ =-\frac{1}{2{\sigma }^2}{\displaystyle \sum _{n=0}^{N-1}{(real(Cor{r}_n)-A_{mp}\cos (\theta ))}^2}-\frac{1}{2{\sigma }^2}{\displaystyle \sum _{n=0}^{N-1}{(imag(Cor{r}_n)-A_{mp}\sin (\theta ))}^2}-N\ln (2\pi {\sigma }^2)\\ =-\frac{1}{2{\sigma }^2}{\displaystyle \sum _{n=0}^{N-1}\{{\left|Cor{r}_n\right|}^2+A_{mp}{}^2-2A_{mp}[real(Cor{r}_n)\cos (\theta )+imag(Cor{r}_n)\sin (\theta )]\}}-N\ln (2\pi {\sigma }^2)\end{array}$$

(13)

where Corr_n = Corr(n), Corr_N = [Corr(0), Corr (1), ……, Corr (N − 1)] is N consecutive coherent integration results. μ = [A_mp Δf Δφ]^T represents the signal parameters to be estimated, n = 0…N − 1, θ = 2πΔfnT_coh + Δφ, $real(·)$ and $imag(·)$ represent the real and imaginary part respectively.

By finding the maximum value of Equation (13) to get the estimated value of μ, which can be expressed as:

$$\frac{\partial \Lambda (\mu |Cor{r}_N)}{\partial \mu }=0$$

(14)

Deriving the partial derivative of μ:

$$\frac{\partial \Lambda }{\partial A_{mp}}=-\frac{1}{{\sigma }^2}{\displaystyle \sum _{n=0}^{N-1}\{A_{mp}-[real(Cor{r}_n)\cos (\theta )+imag(Cor{r}_n)\sin (\theta )]}\}$$

(15)

when $\frac{\partial \Lambda }{\partial A_{mp}}=0$, the estimation value of A_mp is obtained:

$$A_{mp}=\frac{1}{N}{\displaystyle \sum _{n=0}^{N-1}[real(Cor{r}_n)\cos (\theta )+imag(Cor{r}_n)\sin (\theta )]}$$

(16)

The signal amplitude A_mp estimated in the MLE is used to subsequently adjust the observation noise covariance matrix R in the KF. Since the items of |Corr_n|², A_mp² and Nln(2πσ²) in Equation (13) do not affect $\Lambda$ partial derivative of Δf and Δφ respectively, we can remove such irrelevant items and the cost function is simplified as:

$$M={\displaystyle \sum _{n=0}^{N-1}[real(Cor{r}_n)\cos (\theta )+imag(Cor{r}_n)\sin (\theta )]}$$

(17)

where θ = 2πΔfnT_coh + Δφ, $real(·)$ and $imag(·)$ represent the real and imaginary part respectively.

We can find that if θ is obtained, then A_mp can be calculated by Equation (16). According to the relationship between θ, Δf and Δφ, θ can be obtained as long as we get the value of Δf and Δφ. Thus A_mp can be acquired by Equation (16), the rest is to estimate the value of θ, which is equivalent to get the value of Δf and Δφ. Solving the problem of θ in Equation (17), which the essence is to obtain the values of Δf and Δφ. Getting the value of Δf and Δφ can be regarded as the process of solving a two-dimensional optimal problem. Therefore, the MLE solution of Δf and Δφ in θ is transformed to solve the two-dimensional optimal solution by Equation (17).

3.3. LM Algorithm

The LM known as the damped least-squares (DLS) method, is one of the most effective algorithms to solve nonlinear least squares problems, which has both the advantages of gradient descent method and Gauss-Newton algorithm (GNA). The LM uses a gradient to find the maximum (minimum) value and obtains the optimal solution through iterative convergence. The (non-negative) damping factor λ is adjusted at each iteration. When λ is used as a smaller value, bringing the algorithm closer to the GNA, whereas if an iteration gives insufficient reduction in the residual, λ can be increased, giving a step closer to the gradient-descent direction. We use the LM algorithm to iterate to get the frequency and phase values in the paper. The duration of the coherent integration time is 1.6 ms, which is consistent with the LM iteration time. And the loop adjustment period of the carrier tracking loop is 25 ms. Since the LM iteration time is less than the loop adjustment period, the algorithm proposed in this paper can ensure the real tracking. Therefore, the LM algorithm is used to solve the two-dimensional optimal solution in Equation (17). The iterative optimal solution is given by the following equation:

$${\stackrel{\wedge }{\mu }}_{i+1}={\stackrel{\wedge }{\mu }}_i+{(H_i+\lambda )}^{-1}{\mathrm{G}}_i,i=0,1,2\dots$$

(18)

where subscript i represents the number of iterations, ${\stackrel{\wedge }{\mu }}_i$ is an 2 × 1 state vector including Δf and Δφ, λ is an 2 × 2 diagonal matrix, used to ensure that H_i + λ is positive definite and adjust the iterative convergence rate. H_i and G_i is 2 × 2 pseudo-Hessian matrix and 2 × 1 gradient vector respectively. H_i and G_i can be written as:

$$H_i={\left[\frac{{\partial }^{2}M}{\partial {\mu }^2}\right]}_{\mu ={\mu }_i}$$

(19)

$$G_i={\left[\frac{\partial M}{\partial \mu }\right]}_{\mu ={\mu }_i}$$

(20)

where

$$\frac{\partial M}{\partial \mu }={\left[\begin{array}{cc}\frac{\partial M}{\partial \Delta f}& \frac{\partial M}{\partial \Delta \phi }\end{array}\right]}^T,\frac{{\partial }^{2}M}{\partial {\mu }^2}=\left[\begin{array}{cc}\frac{{\partial }^{2}M}{\partial \Delta f^2}& \frac{{\partial }^{2}M}{\partial \Delta f\partial \Delta \phi }\\ \frac{{\partial }^{2}M}{\partial \Delta \phi \partial \Delta f}& \frac{{\partial }^{2}M}{\partial \Delta {\phi }^2}\end{array}\right]$$

(21)

The entire LM algorithm iterative process is shown in Figure 4. The first step is to initialize the value of ${\stackrel{\wedge }{\mu }}_0$ and λ. Since we assume that the carrier is correctly tracked at the beginning then ${\stackrel{\wedge }{\mu }}_0=[00]$ and λ initialized to an experience value. Next, Equations (19) and (20) are utilized to get H_i and G_i. Increasing λ to make H_i + λ positive definite. Then Equation (18) is used to update $\stackrel{\wedge }{\mu }$ and Equation (17) is used to judge whether $M({\stackrel{\wedge }{\mu }}_{i+1})>M({\stackrel{\wedge }{\mu }}_i)$ is satisfied. If not, continue to increase λ. If satisfied, judging the gradient vector G is less than 0.2 or greater than 0.8 (the judgment condition for G is set according to the actual test), then adjust λ according to the conditions respectively satisfied. The physical meaning of adjusting λ is that if the estimated value $\stackrel{\wedge }{\mu }$ is closer to the optimal value of iteration, then increase λ to slow down the convergence rate, otherwise decrease λ to accelerate iterative convergence. The termination condition of the above iterative process is that the value of the gradient vector G falls below predefined threshold (Pre_Thres) or the number of iterations (Iter_Num) exceeds the set maximum (Iter_Max).

3.4. KF Model

MLE can obtain the minimum error variance of parameters without relying on the prior distribution of signal estimation parameters. For MLE, the choice of the number of observations N is significant. The reason is that if a small value of N is adopted, the observed sample value of MLE is small, thus the estimated result cannot reflect the true data very well. Conversely, if the observed value of N is large, excessive observation noise will be introduced, which is not beneficial to obtaining accurate results. Therefore, if we use the estimated frequency directly in weak signal environment, the non-negligible estimated error in the carrier loop will occur, and finally restrict the improvement of tracking sensitivity and accuracy under weak signals. Since KF is the optimal filter in linear discrete systems [33]. KF is adopted in this paper to smooth the estimation of the parameters after a non-linear estimation by MLE. Therefore, for the carrier frequency tracking problem of TC-OFDM system, the combination algorithm of MLE and KF is the optimal solution. The proposed carrier loop structure is shown in Figure 5.

The main idea is to use MLE to get the estimated value of Δf and Δφ, and KF to smooth the MLE parameter estimation error to get more accurate Δf and Δφ. The output value of the KF is used to adjust the carrier NCO. The MLE uses the LM iteration to get the next value of Δf and Δφ. In practical application, the form of KF is closely related to the state equation and observation equation.

The state equation describes the behaviour of the state vector. In order to adjust the carrier loop accurately, the state vector in the selected carrier loop is expressed as:

$$X=[\begin{array}{ccc}\phi & {\omega }_d& {\omega }_d^{\prime }\end{array}]$$

(22)

where φ is the carrier phase estimation value, ω_d = 2πΔf denotes the angular frequency estimation value, ${\omega }_d^{\prime }$ represents the angular frequency rate.

The state equation can be expressed as:

$$X_k=\mathsf{\Phi }{X}_{k-1}+W_{k-1}$$

(23)

where W_k−1 is the input Gaussian white noise with mean zero and variance Q, $\mathsf{\Phi }$ is the state transition matrix and the form can be written as:

$$\mathsf{\Phi }=\left[\begin{array}{ccc}1& T& T^2/2\\ 0& 1& T\\ 0& 0& 1\end{array}\right]$$

(24)

where T = NT_coh is the loop update period.

The observation equation describes the relationship between observations and state vectors. The observation equation can be expressed as:

$$Y_k=H{X}_k+V_k$$

(25)

where H is the observation matrix, the form can be written as:

$$H=\left[\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right]$$

(26)

The value of the rate of change of Doppler in the observation matrix H is a predetermined experienced value. The preset value is reasonable which satisfies the current applications in the TC-OFDM system. And this kind of pre-setting for the rate of change of Doppler is beneficial to us to further study the TC-OFDM receiver algorithm in high dynamic scenario.

V_k is the observed noise, the mean is zero and the variance matrix is R. The phase difference $\Delta \theta$ between the input signal and the local signal can be expanded by the Taylor series:

$$\{\begin{array}{l}\Delta \theta (k+1)=\Delta \theta (k)+T\Delta {\omega }_0(k)+\frac{T^2}{2}\Delta {\omega }_1(k)+{\xi }_1(k)\\ \Delta {\omega }_0(k+1)=\Delta {\omega }_0(k)+T\Delta {\omega }_1(k)+{\xi }_2(k)\\ \Delta {\omega }_1(k+1)=\Delta {\omega }_1(k)+{\xi }_3(k)\end{array}$$

(27)

where $\Delta \theta$ is the carrier phase, $\Delta {\omega }_0$ is the residual carrier and $\Delta {\omega }_1$ is the rate of change of Doppler.

The form of ${\xi }_i$ can be written as:

$$\begin{array}{cc}{\xi }_i(k)={\displaystyle {\int }_{(k-1)T}^{kT}\frac{{\tau }^{3-i}}{(3-i)!}}Y(\tau )d\tau ,& i=1,2,3\end{array}$$

(28)

${\xi }_i(k)$ is the remainder of Taylor expansion, which is used to represent model noise and describes the effects of random interference and model inaccuracy as described above. $Y(t)$ is a zero-mean, white Gaussian noise process with a single-sided spectral density N_y. The noise variance can be expressed as:

$$E[Y^2(t)]={\sigma }_y^2=\frac{N_y}{2T}$$

(29)

and the value of $E[{\xi }_i(k)·{\xi }_j(k)]$ can be obtained:

$$\begin{array}{l}E[{\xi }_i(k)·{\xi }_j(k)]\\ =E\left\{\left[{\displaystyle {\int }_{(k-1)T}^{kT}\frac{u^{3-i}}{(3-i)!}}Y(u)du\right]\left[{\displaystyle {\int }_{(k-1)T}^{kT}\frac{v^{3-i}}{(3-i)!}}Y(v)dv\right]\right\}\\ ={\displaystyle \underset{uv}{\iint }\left\{\frac{u^{3-i}}{(3-i)!}·\frac{v^{3-i}}{(3-i)!}E[Y(u)·Y(v)]\right\}}dudv\\ =\frac{{\sigma }_y^2·T^{8-(i+j)}}{(3-i)!·(3-j)!·[7-(i+j)]}\end{array}$$

(30)

Then the variance matrix Q can be derived:

$$\begin{array}{l}Q=E[\xi (k){\xi }^T(k)]\\ =E\left\{\left[\begin{array}{c}{\xi }_1(k)\\ {\xi }_2(k)\\ {\xi }_3(k)\end{array}\right]\left[\begin{array}{ccc}{\xi }_1(k)& {\xi }_2(k)& {\xi }_3(k)\end{array}\right]\right\}\\ ={\sigma }_w^{2}I\left[\begin{array}{ccc}{T}^4/20& T^3/8& T^2/6\\ T^3/8& T^2/3& T/2\\ T^2/6& T/2& 1\end{array}\right]\end{array}$$

(31)

where ${\sigma }_w^2={\sigma }_y^2{T}^2$.

The forms of Q and R can be respectively denoted as [34]:

$$Q={\sigma }_w^{2}I\left[\begin{array}{ccc}{T}^4/20& T^3/8& T^2/6\\ T^3/8& T^2/3& T/2\\ T^2/6& T/2& 1\end{array}\right]$$

(32)

$$R={\sigma }_v^{2}I$$

(33)

where I is the unit matrix.

The σ_v² in R is adjusted according to the value of A_mp estimated by MLE. For the σ_v² in the observation covariance matrix R, we obtained a series of experience values based on multiple experiments during the algorithm experiments. According to the estimated value of the signal amplitude A_mp obtained in the MLE, these experience values are utilized to adjust the preset value of the σ_v².

The KF algorithm obtains the filter value at the current time according to the state estimation value at the previous moment and the observation value at the current time. The entire carrier loop filtering process can be divided into two parts, the state estimation and state prediction. The output state estimate X_k of KF and the variance matrix P_k of the estimation error can be iteratively calculated by the following equations:

$${\mathrm{X}}_{k,k-1}=\mathsf{\Phi }{\mathrm{X}}_{k-1}$$

(34)

$$P_{k,k-1}=\mathsf{\Phi }{P}_{k-1}{\mathsf{\Phi }}^T+Q_{k-1}$$

(35)

$$K_k=P_{k,k-1}{H}_k^T{[H_k{P}_{k,k-1}{H}_k^T+R_k]}^{-1}$$

(36)

$$X_{k\mathrm{k}}={\mathrm{X}}_{k,k-1}+K_k[Y_k-\mathrm{H}{X}_{k,k-1}]$$

(37)

$$P_{k,k}=[I-K_{k}H]P_{k,k-1}$$

(38)

From Equation (36) can be seen that when R_k is large, the corresponding K_k will be small, then the state estimate calculated by Equation (37) is small; when Q_k is small, one step prediction covariance matrix P_k,k−1 calculated by Equation (35) will be small, which lead to a smaller state estimate X_k−1 finally. From the above analysis, it can be seen that every update of the system state by KF is a compromise between the current system state uncertainty and the observation uncertainty. Therefore, in this paper, R_k and Q_k are obtained by real-time statistics of noise on historical observation and current observation to enhance the adaptability of KF to noise.

4. Simulation and Analysis

According to the previous discussion, a novel carrier loop algorithm based on MLE and KF is designed. In order to further illustrate the feasibility and performance of the proposed algorithm, simulations and real data tests are performed in this section. Monte Carlo simulations are adopted to compare the proposed algorithm with the current algorithms to make a comprehensive evaluation of the novel carrier loop algorithm. In addition, the TC-OFDM signals are broadcasted by the modified base stations, a positioning receiver and other related equipment is also utilized to achieve the proposed algorithm. Finally, we choose several points in a test building to test the positioning accuracy for the static receiver.

4.1. Simulations

In order to verify the performance of the proposed algorithm, Monte Carlo simulation is applied to evaluate the proposed algorithm effectively and comprehensively. To demonstrate the reliability and effectiveness of the novel carrier loop algorithm, comparative tests are performed. The feasibility of iterative estimation of the LM algorithm, the validity of the KF smoothing error and the superiority of the combination the MLE with the KF algorithm are verified, respectively.

4.1.1. Determine the Number of Observations for MLE

In statistics, MLE is a method of estimating the parameters of a statistical model. And the estimation accuracy of parameter using MLE is closely related to the number of signal observations used for one estimation. In general, the accuracy of the MLE parameter estimation increases with the number of observations used. Therefore, it is necessary to first analyze how many observations is suitable for one MLE in the carrier loop. The algorithm proposed uses coherent integration results as the MLE observations. We use the loss of lock probability as a standard reference criteria to evaluate the number of observations. The relationship between loss of lock probability and SNR using different observations is shown in Figure 6. The numbers in the legend in Figure 6 represent the values of N. The positioning signal adopts Gold codes and the simulation parameters are listed in

Table 1. Simulation parameters.

Parameter	Value
Slot time, TF	25 ms
Sample rate	4.4
Intermediate frequency, fIF	0 Hz
Data bit transition	Random
Coherent integration time, Tcoh	1.6 ms
Predefined threshold, Pre_Thres	0.02
Iteration maximum numbers, Iter_Max	15
Signal to noise ratio, SNR	−45 dB–−20 dB

.

Figure 6 shows the correspondence between the loss of lock probability and SNR of the proposed algorithm when N = 15, N = 30, and N = 45. It can be seen from the results that as the number of sample observations (denoted as N) gradually increases, the anti-noise ability of the proposed algorithm gradually increases, which means the loss of lock probability with same SNR gradually decreases. The result in Figure 6 is reasonable. Because the value of SNR in experiments is really low, the influence of noise under weak signals will be greater than the different number value of N, which is the main factor. We took a piece of data at low SNR to illustrate the problem in TC-OFDM system. Also, 500 Monte Carlo simulations are made to give RMS of frequency error with different sample observations. The results are shown in Figure 7.

Figure 7 gives the RMS frequency tracking error performance curve for the proposed algorithm with different N values. The numbers in the legend in Figure 7 represent the values of N. From the results, it can be seen that when the SNR is less than −35 dB, the larger of N, the higher frequency estimation accuracy of the MLE&KF. The reason is that the larger number of sample observations leads to the MLE with higher frequency resolution. However, the noise power gradually decreases with the increase of SNR. When the SNR is greater than −35 dB, an excessively large value of N will lead to excessive steady state error. Therefore, the error curve corresponding to N = 45 will be degraded. In addition, Figure 7 also compares the error performance curve of MLE&KF with EKF. From the comparison results, it can be found that the performance of the frequency tracking error of the MLE&KF designed for the optimal estimation and filtering algorithm is significantly improved than the performance of the EKF based on the sub-optimal filtering algorithm. According to the above analysis, observations with N = 30 are used to make a reasonable statistical sample observations.

4.1.2. LM iteration Effectiveness

To test the feasibility and effectiveness of the MLE and the LM algorithm, we set the residual carrier as 25 Hz and the phase as 0.5 rad of the input TC-OFDM signal. The SNR is −25 dB and other parameters are same as Table 1. The convergence curve of frequency and phase using MLE and iteratively by LM algorithm is shown in Figure 8. From Figure 8, after iteration to the 9th, the residual carrier and phase values remain stable and are close to the set value. The final residual carrier value is stable at 25.8 Hz and the phase value is stable at 0.53 rad, which verifies the feasibility of using MLE to estimate the residual carrier and phase of the carrier loop. For the convenience of analysis, we selected the iteration number fixed at nine in the following tests.

4.1.3. KF Effectiveness of Smoothing Error

Meanwhile, there is still a large frequency error in adjusting the carrier loop directly using the signal parameters estimated by MLE for weak TC-OFDM signals. KF is adopted to smoothing the frequency error, so it is necessary to demonstrate the validity of KF smoothing error, and the MLE combined with KF loop is compared with MLE loop. The SNR is −25 dB and other parameters are same as Table 1. Figure 9 shows that the error of frequency estimation after KF is obviously decreased in contrast to using MLE only, which verified the validity of KF smooth frequency error. The MLE combined with KE denotes as MLE&KF and the other denotes as MLE in Figure 9, respectively.

4.1.4. Performance Comparison Results of the Proposed Algorithm and Other Algorithms

To verify the signal parameter estimation accuracy corresponding to different SNR, we compare the proposed algorithm with the conventional second-order FLL assisted third-order PLL (FLL&PLL) carrier loop and EKF algorithm. The FLL&PLL displays both the high accuracy of PLL and the large dynamics of FLL. The specific parameters and corresponding discriminator algorithm of the second-order FLL assisted third-order PLL are given in

Table 2. The specific parameters and corresponding discriminator algorithms of second-order FLL assisted third-order PLL.

Parameter	Value
PLL bandwidth, Bw,PLL	10 Hz
FLL bandwidth, Bw,FLL	20 Hz
Wn of carrier PLL	Bw,PLL/0.53
Wn of carrier FLL	Bw,FLL/0.53
Phase discrimination equation	$Q_p·sign(I_p)/sqrt(I_p{}^2+Q_p{}^2)$
Frequency discrimination equation	$P_{cross}·sign(P_{dot})/(I_p{}^2+Q_p{}^2)$ where $P_{cross}=I_{p-1}·I_p+Q_{p-1}·Q_p$, $P_{dot}=I_{p-1}·Q_p-Q_{p-1}·I_p$

. The EKF algorithm used for comparison in this paper is a pure EKF algorithm that is independent of MLE. The result of the coherent integration is used as the input observation signal parameter of the EKF. Since the carrier loop can be considered as a nonlinear discrete-time system. The purpose of using the EKF-based carrier loop is to linearize the state equation and the observation equation of carrier tracking loop to obtain a sub-optimal estimation method for the carrier loop. The algorithm proposed in this paper is to use MLE to estimate the interested parameters of the carrier loop, and then use KF to filter estimated parameters. Since the carrier loop can be considered as a nonlinear discrete-time system. The KF is a linear discrete system filter which cannot be directly used to the carrier loop. We use MLE to estimate the carrier loop parameters and do a non-linear approximation, and then use KF for filtering, which is different from the pure EKF algorithm. Two hundred Monte Carlo simulations are performed for each SNR with a fixed predetermined frequency error. The RMSE frequency error comparison results are shown in Figure 10. The results in Figure 10 show that the frequency estimation errors of the three algorithms all increase with SNR decreasing. Among them, the FLL&PLL has the largest error under low SNR, followed by the EKF. The proposed algorithm (MLE&KF) can still effectively estimate the frequency, which shows high frequency estimation accuracy and tracking performance for weak TC-OFDM signals. The simulation verifies the superiority of the proposed algorithm for various SNR.

Meanwhile, in order to further verify the tracking performance of the proposed algorithm, the comparison results of the tracking probability with other two algorithms are shown in Figure 11. Tracking probability is defined as the probability of successfully tracking the total number of Monte Carlo simulations. Tracking sensitivity is defined as the SNR at which tracking probability exceeds 50%. At present, the definition of tracking sensitivity can be divided into two types: the detection probability is 50% [31] and 90% [35]. In order to better illustrate the problem, this paper refers to [31] for the definition of tracking sensitivity. According to the results of the detection probability of the proposed algorithm in Figure 11, the SNR is adopted to be taken as the tracking sensitivity when the tracking probability is 50%, which can reflect the overall level of the performance of the algorithms. It can be seen that the tracking sensitivity of FLL&PLL is −31.1 dB, followed next by EKF which is −33 dB. The MLE&KF algorithm has better performance than the others, and the tracking sensitivity is −35.3 dB, which is 2.3 dB lower than EKF and 4.2 dB lower than FLL&PLL. The Monte Carlo simulation results prove that proposed algorithm can significantly improve the tracking sensitivity for weak TC-OFDM signals.

4.2. Real Data Tests

Real data tests are conducted to prove that the novel carrier loop algorithm has significantly good performance in an actual environment. We start our actual tests in a comprehensive test platform, which is shown in Figure 12 and Figure 13. The test platform consists of a modified base station and the TC-OFDM receiver. Figure 12 describes each component of the modified base station in detail. The time distributor, the counter and the industrial personal computer collaborative work together in order to ensure synchronization between the modified base stations. Each modified base station equipped with an atomic clock with an output frequency of 10 MHz. The synchronization accuracy between the modified base stations is up to 5 ns (1σ). Meanwhile, the time distributor generates the positioning data messages containing UTC, air pressure, base station number and base station coordinates which are necessary for positioning. The function of the actuator is to modulate the TC-OFDM signal into RF signal and finally the RF signal is transmitted by the transmitter.

We use the TC-OFDM receiver developed by us which is shown in Figure 13 to test the positioning accuracy. An overview of the internal and external structures of the TC-OFDM receiver is shown in Figure 13a. Figure 13b shows that the TC-OFDM receiver and mobile phone communicate through the Bluetooth protocol to transmit useful data messages. Then the phone exhibits the final positioning result via the map. An architecture based on FPGA and ARM is adopted in the TC-OFDM receiver for baseband processing and data demodulation. The FPGA is used to perform the main logic operations and ARM is responsible for controlling the circuit logic. The IF signal is processed into a zero-digital IF signal for subsequent FPGA processing. The LM iterative and KF algorithm used in the proposed carrier loop are implemented in the ARM processer. The main computational load is in LM. In each LM iteration, sine, cosine operations and matrix calculations are needed. The above calculations are easily calculated in ARM which can meet real tracking requirements for carrier loop.

In addition, we implement the proposed algorithm in our own developed TC-OFDM receiver. The EKF algorithm and two-order FLL assisted three-order PLL algorithm are also implemented to make a comparison. The performance when the three algorithms track weak TC-OFDM signals can be evaluated in practical applications as shown in Figure 14. The modified base station is utilized to generate TC-OFDM signals. Then the TC-OFDM signal is sent to a signal attenuator. The signal attenuator is used to set different signal power. Then the receiver receives TC-OFDM signal of different signal power for subsequent baseband processing. Finally, the useful data is transmitted to the mobile phone for positioning. We use signal attenuator to gradually reduce the signal power to test the ability of tracking weak signal performance of three algorithms. The test results are shown in

Table 3. Tracking sensitivity of three algorithms under different signal powers.

Signal Power (dBm)	−126	−127	−128	−129	−130	−131	−132	−133	−134	−135	−136
the proposed algorithm	O 1	O	O	O	O	O	O	O	O	O	O
EKF	O	O	O	O	O	O	O	O	O	-	-
FLL&PLL	O	O	O	O	O	O	O	- 2	-	-	-

¹ The mark O represents the algorithm can track the signal at the corresponding signal power. ² The mark—represents the algorithm cannot track the signal at the corresponding signal power.

, where it can be seen that the proposed algorithm has better tracking sensitivity for weak TC-OFDM signals by 2–4 dB. In the real TC-OFDM system, the weak signal power is generally between −125 dBm to −130 dBm. But, under the special cases, the signal power is low to −136 dBm. Therefore, the sensitivity improvement of the proposed algorithm is modest, and it can satisfy the use of existing TC-OFDM systems. Meanwhile, the proposed carrier loop tracking algorithm in this paper is implemented in the receiver’s ARM structure. The ARM development resources of existing TC-OFDM receivers can meet the complexity and computational burden of the proposed algorithm. Furthermore, from Table 3, the proposed algorithm can effectively track the TC-OFDM signal when the signal power is −136 dBm. The signal power using the algorithm of FLL-assisted PLL and pure EKF is −132 dBm and −134 dBm, respectively. For signals with lower signal power, these two algorithms can no longer track steadily, and even lose lock, so the improvement of signal tracking sensitivity of the proposed algorithm is significant for practical applications.

Furthermore, we ran an actual test on the campus suing the test environment shown in Figure 15. We set up the base station on the roof of four buildings on our campus. Then we selected the 3th floor of another building as test building to start our real test using the TC-OFDM receiver. The two-order FLL assisted three-order PLL and EKF algorithms are implemented in the receiver to compare with the proposed algorithm. In order to verify the better performance of the proposed algorithm for weak TC-OFDM signal, we selected 30 test points on the floor of the test building for positioning accuracy test. The receiver is placed at each test point for ten minutes, then we calculated the Root-mean-square error (RMSE) of the test points. The corresponding acquisition and positioning algorithms are utilized for the horizontal positioning. Since the positioning system uses a custom coordinate system for indoor positioning, the output positioning result is compared with the distance between the selected point and the origin of the corresponding floor. The RMSE of the positioning accuracy error results of the three algorithms between the selected point and the corresponding original point are shown in Figure 16, which shows the horizontal measurement accuracy results of three algorithms. The horizontal axis in the figure represents the sequence number corresponding to the selected 30 points. The vertical axis represents the calculation error of the final positioning result corresponding to each selected point. The positioning results of the blue, red, and black lines respectively represent the three different carrier loop algorithms: FLL&PLL, EKF, and MLE&KF. The final positioning results of using MLE&KF has the smallest fluctuations and the best positioning accuracy, which shows the superior performance of our algorithm compared to other algorithms. Also, these results is verified the effectiveness of the proposed algorithm in practical applications. The horizontal positioning accuracy is better than 3 m. The TC-OFDM receiver can be used to achieve accurate positioning in many indoor LBS applications, such as indoor mall pedestrian navigation and positioning and elderly care terminal. It can be seen that the positioning accuracy obtained by the proposed algorithm has smaller positioning error and higher positioning accuracy than the others for weak TC-OFDM signals.

5. Conclusions

To improve the tracking performance for weak TC-OFDM signals, a novel carrier loop algorithm based on MLE and KF is proposed in this paper. The algorithm derives the MLE discriminator function to replace the existing discriminator in carrier loop. The Levenberg-Marquardt (LM) algorithm is utilized to obtain the MLE cost function consisting of signal amplitude, residual carrier frequency and carrier phase, and the MLE discriminator function is derived from the corresponding MLE cost function. In order to further reduce the residual frequency error, the KF is used to smooth the MLE discriminator function results. Numerical simulation and real data tests are implemented to verify the performance of the algorithm. The test results indicate that the novel carrier loop algorithm can improve the tracking sensitivity of the TC-OFDM receiver by taking full advantage of the characteristics of the carrier loop parameters. Finally, compared with the current carrier loop algorithm, the tracking sensitivity is effectively improved by 2–4 dB and the positioning accuracy is experimentally improved in the real environments.