Radar Waveform Selection for Maneuvering Target Tracking in Clutter with PDA-RBPF and Max-Q-Based Criterion

Abstract

In this paper, to track maneuvering unmanned surface vehicles (USVs) in scenarios with clutter, we propose a novel method based on the probabilistic data association (PDA) algorithm and Rao-Blackwellized particle filter (RBPF) algorithm, and we further improve the tracking performance by Max-Q criterion-based waveform selection. This work develops a maneuvering target model in the context of clutter, integrating linear and nonlinear states as well as observations with false alarms. In order to jointly tackle the mixed-state tracking problem, the PDA algorithm is integrated into the RBPF framework. This allows it to be used with the complex nonlinear and linear hybrid system and helps to minimize the state dimensions of conventional particle filtering (PF). Additionally, by utilizing Q-learning principles, we provide a Max-Q-based criterion to select the waveform parameters, which guarantees low measurement errors and efficiently handles measurement uncertainties. Our simulation results show that the PDA-RBPF algorithm, which has a more appropriate tracking mechanism, produces results that are more accurate than those of the EKF or PF algorithms alone. Furthermore, the RMSE derived by the Max-Q-based criterion is smaller and more robust than that of other selection methods, as well as yielding a fixed waveform. Our proposed mechanism, which combines the concepts of PDA-RBPF and Max-Q waveform selection, performs well in target tracking tasks and exhibits relatively good performance over some existing ones.

1. Introduction

Currently, in the tracking of maneuvering USVs, traditional radar with single waveforms cannot adapt well to the complex and dynamic marine scenario. As discussed in [1,2,3], modern radar has focused on designing agile waveforms to enhance the tracking performance, where the tracking error is typically represented by the state estimation error covariance matrix and also depends on the transmitting waveform. Generally, by predefining a waveform library, the criterion for waveform selection is to optimize the cost function of different parameters at each time instant [2,3], where the goal is to select the waveform with the lowest cost value as the optimal transmitting waveform. Therein, such criteria include the minimum mean square error criterion (Min-MSE) [4], the maximum mutual information criterion (Max-MI) [5,6] and the minimum gate criterion (Min-Gate) [7]. A series of simulations and results show that these criteria-based optimization methods have high tracking accuracy and simple calculation principles, but with relatively high time costs [8]. Reference [9] proposed a new method using the particle swarm optimization (PSO) algorithm based on adaptive Kalman filtering to optimize the radar waveform parameters. The proposed method can reduce the velocity error and range error by 50% and 60%, respectively. The authors in [10] proposed an adaptive waveform selection algorithm based on indirect reinforcement learning, addressing uncertainties in the target-state space. Simulation results show that the algorithm has better computational efficiency and fewer state estimation errors, which improve the tracking accuracy. The authors in [11] also proposed a universal reinforcement learning waveform selection strategy to solve a broad class of waveform-agile tracking problems, while making minimal assumptions about the environment’s behavior. Therefore, their strategy is more general, and it provides better performance. Moreover, the authors in [8,12] utilized supervised learning and reinforcement learning for waveform selection, where these adaptive waveform selection methods further reduce the state estimation error and improve the tracking accuracy with much less processing time compared to the criteria-based optimization methods.

In general, most studies have focused on the cost function and choosing the criterion and learning mechanism, and they have ignored the effect of the target model on the receiver’s estimation filter. For a simple scenario (especially with a linear observation model), the effective Kalman filtering method has been proven to be the best [13]. Meanwhile, for complex marine scenarios with nonlinear non-Gaussian models, the traditional Kalman filter seems to be insufficient. Here, most marine USV tracking scenarios can be formulated as nonlinear and non-Gaussian or more sophisticated models. The estimation methods for such nonlinear models include the extended Kalman filter (EKF), particle filter (PF) and so on [14,15,16]. The limitation of the EKF is that it is only suitable for local linearization, aimed at models with a low degree of nonlinearity. Moreover, too many particles in the PF leads to higher computational complexity [17,18]. The question of how to balance the linear and nonlinear models is a problem that deserves careful consideration. In order to solve the uncertainty of the maneuvering model, authors usually turn to interacting multiple model (IMM) algorithms [7,19]. These algorithms result in remarkable improvements in the state estimation accuracy but greatly increase the computational workload [20]. To address this urgent issue, the Rao-Blackwellized particle filter (RBPF) algorithm can divide the state space into two subspaces [21,22], where one subspace is updated using a PF and the other employs other filters [23]. Due to the separation of the state space, the RBPF algorithm reduces the dimensionality of nonlinear states by the PF to ensure the accuracy of nonlinear filtering. Compared with traditional PF algorithms, given the same number of particles, the RBPF reduces the computational complexity and exhibits superior performance [24].

Based on the RBPF’s traits, in this paper, we further improve the tracking performance by utilizing the probabilistic data association (PDA) algorithm, Rao-Blackwellized particle filter (RBPF) algorithm and Max-Q-based adaptive waveform selection principle. We establish a hybrid system incorporating both nonlinear and linear components, considering some false alarms and clutter data. The PDA and RBPF algorithms are combined to address clutter interference and estimate the target’s state. Moreover, by leveraging the mechanism of Q-learning, we adaptively select the optimal waveform candidates from the waveform library in order to minimize the estimation errors and improve the tracking accuracy. Figure 1 shows the radar system for maneuvering target tracking in clutter scenario.

2. System Overview and Problem Formulation

In this section, we consider a radar system consisting of a transmitter located at $t$ and a receiver located at $r$. Differing from the discussion in [8], the receiver in our work uses a novel PDA-RBPF algorithm to obtain local tracking trajectories. Thus, the dynamic hybrid model of the maneuvering target can be defined as

$$\left\{\begin{array}{c}{\mathit{x}}_k^n=\mathit{f}({\mathit{x}}_{k-1}^n)+{\mathit{w}}_k^n\\ {\mathit{x}}_k^l=\mathit{F}{\mathit{x}}_{k-1}^l+{\mathit{w}}_k^l\\ z_k=h(x_k)+u_k\end{array}\right.,$$

(1)

where $x_k^n$ and $x_k^l$ denote the nonlinear subsection and linear subsection of the target state vector at time instant $k$, respectively. $f(\cdot )$ represents the transition function of the nonlinear state. $F$ represents the linear state transition matrix. $w_k^n$ and $w_k^l$ are the nonlinear and linear Gaussian process noise at time instant $k$, respectively. $z_k$ is the state measurement value at time instant $k$. $h(\cdot )$ denotes the nonlinear function of the observation process. Finally, $u_k~N(0,R)$ is the measurement noise.

Generally, the target state vector at time instant $k$ can be denoted as $x_k={[p_{x,k}, p_{y,k}, v_{x,k}, v_{y,k}]}^T$, incorporating the position part $p_k={[p_{x,k}, p_{y,k}]}^T$ and velocity part $v_k={[v_{x,k}, v_{y,k}]}^T$. Assuming that the target is located at $p_k$ with velocity $v_k$, and the time delay ${\tilde{\tau }}_k$ and Doppler shift measurements ${\tilde{\zeta }}_k$ are available for the receiver, then the radial range and range rate measurements can be calculated as

$$d_k=‖p_k-r‖,$$

(2)

$${\rho }_k=\frac{(p_{x,k}-r_x)v_{x,k}}{‖p_k-r‖}+\frac{(p_{y,k}-r_y)v_{y,k}}{‖p_k-r‖},$$

(3)

where $r_x$ and $r_y$, respectively, represent the x and y positions of the transmitter.

By assembling (2) and (3) together, the measurement vector has

$${\tilde{\mathit{z}}}_k={[{\tilde{d}}_k,{\tilde{\rho }}_k]}^T=z_k+e_k={[d_k,{\rho }_k]}^T+{[e_{d,k},e_{\rho ,k}]}^T.$$

(4)

The range rate measurement error covariance matrix has

$$R_k({\psi }_k)={\Gamma }_{\{\tau ,\zeta \}}{C}_{\{\tau ,\zeta \}}({\eta }_k,{\psi }_k){\Gamma }_{\{\tau ,\zeta \}},$$

(5)

where ${\Gamma }_{\{\tau ,\zeta \}}=diag\left\{c,c/f_0,1\right\}$ is the transition matrix. As discussed in [1,25], $C_{\{\tau ,\zeta \}}({\eta }_k,{\psi }_k)$ is the Cramér–Rao Lower Bound (CRLB) matrix with a time delay and Doppler shift depending on SNR $\eta$, as well as the transmitted waveform parameters. Although $R_k$ is also dependent on SNR ${\eta }_k$, here, we simply treat it as a function of waveform parameter vector ${\psi }_k$ and focus on the adaptive waveform selection. Furthermore, the CRLB for the measurement error covariance of the Gaussian frequency-modulated waveform can be obtained by

$$R(\psi )=\left[\begin{array}{cc}{c}^2{\lambda }^2/(2\eta )& -c^{2}b{\lambda }^2/(f_0\eta )\\ -c^{2}b{\lambda }^2/(f_0\eta )& c^2(1/(2{\lambda }^2)+2b{\lambda }^2)/(f_0\eta )\end{array}\right].$$

(6)

It can be observed that the measurement error covariance is related to both the pulse duration and the linear frequency modulation rate. Therefore, by adjusting the waveform parameters according to certain metrics, the measurement error covariance $R(\psi )$ can be further reduced, effectively improving the tracking accuracy.

3. Target Tracking Model and Its Algorithm Framework

In practical scenarios, the most crucial issue is how to handle the uncertainty in originating radar echoes and further accurately extract the target information [26]. The most common approach to addressing the problem of maneuverable target tracking with clutter interference is the probabilistic data association (PDA) algorithm. The PDA algorithm assumes that as long as there is a valid echo (within the gate) that may come from the target, each measurement would have a different probability of originating from this target [25]. Then, the PDA algorithm calculates the weighted coefficients for each probability. Finally, the updated target state can be obtained by using the resulting weighted measurements.

It is known that PF is computationally intensive and inefficient when sampling in a high-dimensional state space [27]. High-dimensional state spaces can be classified with different subspace processing methods using different filtering methods. For instance, the linear component can undergo Kalman filtering to derive a conditional posterior distribution, while the nonlinear component can be processed via PF. This hybrid approach yields a mixed filter, effectively reducing the dimensionality of the particle filtering sampling space. In the system model, the state model is represented by $p(x_k|x_{k-1})$, and the observation model is described by $p(z_k|x_k)$. To track a target using RBPF, the specific steps involve the separate application of the PF and KF algorithms for measurement updates and target state prediction updates. This process aims to obtain the state estimation to generate the target trajectory and the covariance matrix of estimation errors for waveform selection.

Combining PDA and RBPF together could further address the tracking problem of mixed state targets in the presence of clutter interference. By marginalizing the linear state vector, particles only need to be present in the low-dimensional nonlinear state space, resolving the high computational complexity of the traditional PF. Meanwhile, KF with the optimal linear filtering method can enhance the tracking accuracy.

The algorithm framework is illustrated in Figure 2, with the detailed steps as follows.

• Initializing particles and parameters

First, set the number N of particles required for PF and then initialize these particles‘ values and other relevant parameters.

• Kalman measurement updating for the linear state

Given the state estimate $x_{k-1}$ and its covariance $P_{k-1}$ at time instant $k-1$, the predicted state estimate and covariance at time instant k can be computed by

$$\left\{\begin{array}{c}{\overline{\mathit{x}}}_k^l=F{x}_{k-1}^l\\ {\overline{P}}_k^l=F{P}_{k-1}^l{F}^{\mathrm{T}}+Q_k\end{array}\right..$$

(7)

• Calculating the predictive measurement

Nonlinear predicted state ${\overline{\mathit{x}}}_k^n=f(x_{k-1}^n)+w_k^n$ and linear predicted state ${\overline{\mathit{x}}}_k^l$ form the target predicted state ${\overline{\mathit{x}}}_k=\left[\begin{array}{c}{\overline{\mathit{x}}}_k^n\\ {\overline{\mathit{x}}}_k^l\end{array}\right]$. The predictive measurement is

$${\overline{\mathit{z}}}_k=h({\overline{\mathit{x}}}_k).$$

(8)

• Determining whether a measurement falls within the validation region

The region around the predictive measurement ${\overline{\mathit{z}}}_k$ is defined as a validation region at time instant $k$. The measurement derived from the target falls within this region with probability $P_G$, which usually refers to the gating probability. The condition for a point $\xi$ to be in the validation region would be satisfied if

$${(\xi -{\overline{\mathit{z}}}_k)}^T{S}_k^{-1}(\xi -{\overline{\mathit{z}}}_k)\le g^2,$$

(9)

where $g$ is a threshold related to the gating probability $P_G$. Here, $S_k$ is the residual covariance matrix.

$$S_k=H_k{\overline{P}}_k^l{H}_k^T+R_k.$$

(10)

• The probability of a validated measurement from the target and its association measurement

Suppose that there are m validated measurements; the probability of the j-th validated measurement ${\overline{z}}_{k,j}$ has

$${\beta }_j=\frac{e_j}{b+{\displaystyle {\sum }_{l=1}^m{e}_l}},j=1,\dots ,m.$$

(11)

The probability of having no measurement from the target falling within the validation region is

$${\beta }_0=\frac{b}{b+{\displaystyle {\sum }_{l=1}^m{e}_l}}.$$

(12)

where $e_j$ is given by

$$e_j=\exp \{-\frac{1}{2}{\upsilon }_{k,j}^T{S}_k^{-1}{\upsilon }_{k,j}\},$$

(13)

where ${\upsilon }_{k,j}$ is the residual for the j-th validated measurement ${\overline{\mathit{z}}}_{k,j}$,

$${\upsilon }_{k,j}={\overline{\mathit{z}}}_{k,j}-{\overline{\mathit{z}}}_k.$$

(14)

The false alarms are assumed to be uniformly spatially distributed over the measurement space and independent over time. A Poisson distribution is used to model the number of false alarms [1], so $b$ is computed as

$$b=\varsigma {\left|2\pi S_k\right|}^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.}\frac{(1-P_{D,k}{P}_G)}{P_{D,k}}$$

(15)

where $\varsigma$ is the clutter density. The detection probability of the target at time instant $k$ is given by $P_{D,k}=P_F^{1/(1+{\eta }_k)}$, and $P_F$ is the probability of a false alarm.

Therefore, the association measurement can be computed by

$$z_k={\displaystyle \sum _{j=1}^m{\beta }_j{\overline{\mathit{z}}}_{k,j}}.$$

(16)

• Normalizing the importance weights

Assuming that the importance distribution is only related to the state and measurement values at the previous moment, we can calculate the importance weights by

$$w_k^{(i)}=w_{k-1}^{(i)}\frac{p(\left.z_k\right|x_k^{(i)})p(\left.x_k^{(i)}\right|x_{k-1}^{(i)})}{q(\left.x_k^{(i)}\right|x_{k-1}^{(i)},z_k)}=p(z_k\left|x_{0:k}^{n,(i)},z_{1:k-1}\right.).$$

(17)

$${\tilde{w}}_k^{(i)}=\frac{w_k^{(i)}}{{\displaystyle \sum _{j=1}^N{w}_k^{(j)}}}.$$

(18)

• Calculating the number of effective particles

The number of effective particles is calculated to quantify the degree of particle weight degeneracy and determine whether resampling is needed.

$$N_{eff}=\frac{N}{1+\mathrm{var}({\tilde{w}}_k^{(i)})}.$$

(19)

In the above equation, the smaller the effective number of particles—i.e., the larger the variance of the weights, indicating a greater disparity between particles with large and small weights—the more severe the weight degeneracy. In practical calculations, the effective number of particles can be approximated by

$$N_{eff}\approx \frac{1}{{\displaystyle \sum _{i=1}^N{({\tilde{w}}_k^{(i)})}^2}}.$$

(20)

Based on (20), we set a threshold value $\zeta$. If the effective number of particles is less than this threshold, $N_{eff}<\zeta$, it indicates severe degeneracy. The resampling mechanism should be applied to control the particles’ situation.

• Particle filter state updating for the nonlinear state

The particle distribution is approximated by

$$x_k^{n,i}~p(\left.x_k^n\right|z^k)={\displaystyle \sum _{i=1}^N{\tilde{w}}_k^{(i)}\delta (x_k-x_k^{(i)})}.$$

(21)

• Kalman state updating

$$\left\{\begin{array}{c}{x}_k^l={\overline{\mathit{x}}}_k^l+K_k{\upsilon }_k\\ P_k^l={\beta }_0{\overline{P}}_k^l+(1-{\beta }_0)P_k^c+{\tilde{P}}_k\end{array}\right.,$$

(22)

where the Kalman gain and combined residual are calculated by

$$K_k={\overline{P}}_k^l{H}_k^T{S}_k^{-1},$$

(23)

$${\upsilon }_k={\displaystyle \sum _{j=1}^m{\beta }_j}{\upsilon }_{k,j},$$

(24)

respectively, and

$$P_k^c={\overline{P}}_k^l-K_k{S}_k{K}_k^T,$$

(25)

$${\tilde{P}}_k=K_k\left({\displaystyle \sum _{j=1}^m{\beta }_j}{\upsilon }_{k,j}{\upsilon }_{k,j}^T-{\upsilon }_k{\upsilon }_k^T\right)K_k^T.$$

(26)

In this way, the proposed PDA-RBPF algorithm can be used to calculate the probability of valid measurements and the importance weight of each particle and then estimate the mixed nonlinear and linear states at each time instant.

4. Adaptive Waveform Selection Mechanism and Discussion

As the measurement error covariance matrix $R_k({\psi }_k)$ is a function of ${\psi }_k$ (waveform parameter vector at time instant $k$), then the error covariance matrix $P_k$ and the residual covariance matrix $S_k$ are also dependent on ${\psi }_k$, as demonstrated in Section 3. Next, we discuss several criterion-based optimization methods.

4.1. Criterion-Based Optimization Methods

4.1.1. Minimum Mean Square Error Criterion

The minimum mean square error method selects waveform parameters based on the mean square error (MSE) of target state estimation at each time instant. A smaller MSE indicates a closer estimation to the true value, resulting in more accurate tracking. For this case, the problem of adaptive waveform selection can be described as minimizing the trace of the error covariance matrix:

$${\psi }_{k+1}^{opt}=\underset{\psi \in \Psi }{\arg \min }\left\{trace(P_{k+1}(\psi ))\right\}$$

(27)

where $\Psi$ denotes the waveform library in the transmitter.

4.1.2. Maximum Mutual Information Criterion

The mutual information between the predicted value ${\overline{x}}_{k+1}$ and the measurement value $z_k$ can represent the certainty degree of one-step prediction for target state estimation [6]. According to the maximum mutual information criterion, the problem of the optimal waveform parameters is formulated as

$$\begin{array}{l}{\psi }_{k+1}^{opt}=\underset{\psi \in \Psi }{\arg \max }\left\{I({\overline{x}}_{k+1};z_k)\right\}\\ =\underset{\psi \in \Psi }{\arg \max }\{\frac{1}{2}log\left[det\left(I+R_k^{-1}{\overline{P}}_{k+1}\right)\right]\\ =\underset{\psi \in \Psi }{\arg \min }\{\det (P_{k+1}(\psi ))\}\end{array}$$

(28)

By using this criterion, the selection of transmitted waveforms can be achieved by minimizing the determinant of the estimated error covariance matrix.

4.1.3. Minimum Gate Criterion

Considering radar measurements in clutter environments, the gating criterion function controls the size of the range gate to reduce false measurements in high-density clutter or measurement errors in high-noise situations, thereby improving the tracking performance. Since the gate volume is proportional to the determinant of the residual covariance matrix, the selection of the optimal waveform parameter can be expressed as

$${\psi }_{k+1}^{opt}=\underset{\psi \in \Psi }{\arg \min }\left\{\det (S_{k+1}(\psi ))\right\}.$$

(29)

4.2. The Proposed Max-Q-Based Criterion

Q-learning is a model-free method that updates the Q-table by calculating the reward at each step and selecting the optimal path to complete the specified task [8]. The Q-value in the Q-table represents the cumulative discounted reward, indicating the expected return when taking action $a(a\in A)$ in state $s(s\in S)$ at a specific moment. The environment provides feedback in terms of rewards $r$ based on the agent’s actions. The main idea of this algorithm is to construct a Q-table that stores the Q-values for each state action and then select actions to maximize the expected return based on these Q-values.

$$Q_{k+1}(s,a)=Q_k(s,a)+\alpha [r_k+\gamma \underset{a^{\prime }}{\max }{Q}_k(s^{\prime },a^{\prime })-Q_k(s,a)],$$

(30)

where $s(s\in S)$ represents the agent state, $a(a\in A)$ represents the action, $\alpha$ is the learning rate, and $\gamma$ is the discount factor.

Therein, the current Q-value remains a proportion of old values, along with the reward obtained from the current action (waveform parameter), representing both the real-time reward of the action and the potential future rewards. By using these principles of Q-learning, we can design a waveform parameter selection mechanism to choose the optimal parameters. Here, we refer to the waveform selection idea as the Max-Q-based criterion.

First, we calculate the entropy state $E{S}_k$ based on the estimated error covariance $P_k$ at time instant $k$. This serves as an evaluation criterion for the radar tracking performance, measuring the overall uncertainty in the estimation of the target’s state.

$$E{S}_k=\det (P_k)$$

(31)

By comparing the entropy state between time instants $k-1$ and $k$, we further provide a real-time reward for the waveform parameters, where the real-time reward function is denoted as

$$r_k=\log (1+\left|E{S}_{k-1}-E{S}_k\right|)sign(E{S}_{k-1}-E{S}_k).$$

(32)

Generally, a decrease in entropy indicates that the waveform factor can reduce the estimation uncertainty and result in a positive reward. Conversely, an increase in entropy implies an increase in uncertainty, leading to a negative reward or a penalty. After obtaining the real-time reward, we update $Q$ based on Formula (30). Finally, we select the waveform parameters corresponding to the maximum Q-value as the optimal decision, which yields the optimal waveform selection strategy for time instant $k+1$. Therefore, the adaptive waveform selection problem can be described as

$${\psi }_{k+1}^{opt}=\underset{\psi \in \Psi }{\arg \max }\left\{Q_{k+1}(\psi )\right\},$$

(33)

where $\Psi$ denotes the waveform library, containing different waveform categories or parameters.

The block diagram in Figure 3 demonstrates the Max-Q-based waveform selection mechanism. After the filtering tracking, the obtained target state estimation and error covariance matrix are fed back to the transmitter. By traversing the waveform library, the next prediction error covariance matrix is calculated, alongside acquiring the entropy state and reward function. Consequently, the cumulative discounted reward Q-value for each waveform is obtained to update the Q-table. The waveform parameters corresponding to the largest Q-value in the Q-table are selected as the best transmitted waveform, which will be transmitted in the next step.

5. Simulations and Analysis

In this section, we consider a tracking scenario in which the radar is located at the origin (0, 0) m. The target’s state information includes the two-dimensional position and velocity, i.e.,

$$x_k={[p_{x,k},p_{y,k},v_{x,k},v_{y,k}]}^T.$$

(34)

The initial position of the target is located at (3000, 3000) m, and the initial velocity magnitude is (2, 2) m/s. It obeys a dynamic hybrid model of a discrete-time system. In this context, the updating of the x-axis position $p_{x,k}$ refers to a nonlinear model, while the y-axis position $p_{y,k}$ and velocities $v_{x,k},v_{y,k}$ adhere to a linear model. Therefore, the state space is divided into two parts, and nonlinear filtering (particle filtering) and linear filtering (Kalman filtering) are performed separately for each part. The measurement vector for the target state can be denoted as $z_k={[p,v]}^T$, and the measurement function is denoted as $h(\cdot )={[h_p(\cdot ),h_v(\cdot )]}^T$, as follows:

$$\left\{\begin{array}{c}p=h_p(x_k)=\sqrt{{(p_{x,k}-r_{x,k})}^2+{(p_{y,k}-r_{y,k})}^2}\\ v=h_v(x_k)=[p_{x,k}-r_{x,k},p_{y,k}-r_{y,k}]{[v_{x,k} v_{y,k}]}^T/p\end{array}\right.,$$

(35)

where p and v represent the measured values for the distance and velocity, respectively. Moreover, ${[r_x r_y]}^T$ represents the position of the receiver.

The initial state of the target has

$$x_0={[3000,3000,2,2]}^T.$$

(36)

The initial error covariance matrix is

$$P_0=diag[{10}^2,{10}^2,2^2,2^2].$$

(37)

The measurement error covariance matrix is

$$R(p,v)(\psi )=\left[\begin{array}{cc}{c}^2{\lambda }^2/(2\eta )& -c^{2}b{\lambda }^2/(f_0\eta )\\ -c^{2}b{\lambda }^2/(f_0\eta )& c^2(1/(2{\lambda }^2)+2b{\lambda }^2)/(f_0\eta )\end{array}\right]$$

(38)

In this simulation, the radar transmitter emits Gaussian LFM pulses, which are given by

$$\tilde{s}(t)={(\frac{1}{\pi {\lambda }^2})}^{1/4}\exp (-(\frac{1}{2{\lambda }^2}-j2\pi b)t^2)$$

(39)

where $\lambda$ is the Gaussian pulse length parameter and $b={\Delta }_F/(2T_s)$ is the frequency modulation rate. ${\Delta }_F$ is the sweep frequency. $T_s$ is the effective pulse duration, which is approximated by $7.4338\lambda$. The transmission frequency is set to $f_0=12\mathrm{GHz}$, the pulse repetition interval to $T=0.5\mathrm{s}$, and the signal propagation speed to $c=3\times {10}^8\mathrm{m}/\mathrm{s}$ [28].

Here, we assume that the waveform library is composed of Gaussian LFM pulses with different pulse lengths and sweep frequencies, where the pulse length ranges within $\lambda =[0.1:0.05:1]\ast {10}^{-6}\mathrm{s}$ and the sweep frequency ranges within ${\Delta }_F=[-1.5:0.2:1.5]\ast {10}^3\mathrm{MHz}$. Random variables falling within the validation region are seen as clutter and added to the predicted measurement (namely, false alarms), where the number of false alarms obeys a Poisson distribution. The gating probability is $P_G=0.9997$, the threshold corresponding to the gate probability is $g=4$, the probability of false alarms is $P_F=0.01$, and the clutter density is $\varsigma =\frac{2f_0}{c^2}$ [1,29].

To analyze the performance of different waveform selection criteria and filtering algorithms, the following performance metrics have been defined, i.e.,

$$RMSE(k)=\sqrt{\frac{1}{4}[{(p_{x,k}-{\widehat{p}}_{x,k})}^2+{(p_{y,k}-{\widehat{p}}_{y,k})}^2+{(v_{x,k}-{\widehat{v}}_{x,k})}^2+{(v_{y,k}-{\widehat{v}}_{y,k})}^2]}$$

(40)

where ${\widehat{p}}_{x,k}$, ${\widehat{p}}_{y,k}$, ${\widehat{v}}_{x,k}$, and ${\widehat{v}}_{y,k}$ represent the true state values of the target model. Additionally, the average value of the RMSE (ARMSE) according to the position variable is

$$ARMS{E}_{px}=\sqrt{\frac{1}{T}{\displaystyle \sum _{k=1}^T{(p_{x,k}-{\widehat{p}}_{x,k})}^2}}.$$

(41)

The ARMSEs for other state components are calculated in the same way.

We first consider the tracking problem with clutter data by using the PDA algorithm and then proceed to compare the RBPF algorithm with other nonlinear tracking algorithms.

In this section, the nonlinear target’s x-axis position is represented by the inverse tangent function and is also affected by the x-axis velocity, i.e.,

$$p_{x,k}=\arctan (p_{x,k-1})+p_{x,k-1}+2\times v_{x,k-1}.$$

(42)

The y-axis position $p_{y,k}$ and velocities $v_{x,k}, v_{y,k}$ adhering to a linear model are

$$\left\{\begin{array}{c}{p}_{y,k}=p_{y,k-1}+v_{x,k-1}\\ v_{x,k}=0.92\times v_{x,k-1}-0.1\times v_{y,k-1}\\ v_{y,k}=0.1\times v_{x,k-1}+0.92\times v_{y,k-1}\end{array}\right.$$

(43)

As shown in Figure 4 and Figure 5, the RMSE of the EKF algorithm is significantly greater than that of the other two algorithms. The RMSE of the PF algorithm is below 5, while the RMSE of the RBPF algorithm can reach about 1. It can be observed that the RBPF algorithm results in a smaller RMSE compared to the EKF or PF algorithm alone, indicating a more accurate result. The reason is that KF in the RBPF framework is the optimal estimation algorithm for the linear state updating model, while PF provides a more accurate estimation for the complex nonlinear part. The combination of the two parts can improve the accuracy of target tracking.

Next, the PDA-RBPF algorithm and Max-Q-based criterion are used to estimate the target trajectory and select waveform parameters. We set the number of particles N = 100. In Figure 6 and Figure 7 and

Table 1. ARMSE comparison using different waveform selection mechanisms in PDA-RBPF.

ARMSE	${\mathit{p}}_{\mathit{x}}(\mathit{m})$	${\mathit{p}}_{\mathit{y}}(\mathit{m})$	${\mathit{v}}_{\mathit{x}}(\mathit{m}/\mathit{s})$	${\mathit{v}}_{\mathit{y}}(\mathit{m}/\mathit{s})$
Max-Q	0.9738	0.9536	0.4453	0.6590
Fixed-waveform	1.5853	1.5923	0.4629	1.2830
Min-MSE	1.2831	1.2490	0.4552	0.8850
Max-MI	1.5042	1.4540	0.4544	1.0707
Min-Gate	1.2772	1.1897	0.4836	1.1497

, we demonstrate a series of comparisons using different waveform selection schemes. Based on the PDA-RBPF algorithm for target tracking, the waveform parameters are selected using the Max-Q-based criterion proposed in this paper and compared with fixed waveform parameters and other criterion-based optimization methods. The RMSE and ARMSE of target state estimation are listed.

In Figure 6, it is evident that the target undergoes maneuvering motion, where the first obvious turn occurs around $t=5~10s$ and the second large turn occurs around $t=15s$. During turning, the direction of the x-axis or y-axis velocity often changes. Correspondingly, the RMSEs in these turning instances exhibit varying tendencies, as shown in Figure 7. In particular, the increase in the RMSE is particularly noticeable during the large turn of the target’s movement around $t=15s$. It is noted that the utilization of the PDA-RBPF algorithm and the Max-Q criterion can maintain the RMSE at each time instant within 2 or even blow 1. The ARMSEs for the position and velocity estimations are around 1 m and 0.5 m/s, respectively. Compared with other waveform selection methods, the RMSE obtained by the Max-Q-based criterion is smaller and more stable. Our proposed algorithm demonstrates excellent tracking performance.

Additionally, the optimal waveform parameters $\lambda$, ${\Delta }_F$ for the transmitting waveform at each time instant are adaptively selected according to different methods, as shown in Figure 8.

In Figure 8, we can find that the waveform parameter selection process also fluctuates with the target’s maneuvering state. This phenomenon means that different waveform selection schemes often choose different waveform parameters due to the use of their own criteria and produce different tracking results. Obviously, our proposed Max-Q method can adapt to the maneuvering state and adjust to reduce the tracking ARMSE, as shown in Table 1.

6. Conclusions

In this paper, we propose a novel waveform selection mechanism based on the Max-Q criterion using PDA-RBPF for maneuvering target tracking in clutter. Therein, the PDA-RBPF algorithm separates the linear and nonlinear states from the complex mixed system and employs Kalman filtering and particle filtering to estimate the target’s states, including the position and velocity. This algorithm is not only suitable for complex hybrid systems but also reduces the computational complexity and improves the tracking accuracy through appropriate filtering methods. By maximizing the Q-value of the Max-Q-based criterion, we can adaptively select the waveform parameters to effectively reduce the tracking errors and improve the tracking precision. The simulation results demonstrate that the proposed algorithm can adjust the transmitting waveform parameters, resulting in a lower and more stable RMSE compared with a fixed waveform or other selection methods. Moreover, it performs better in tracking a maneuvering target with clutter than other tracking algorithms, equipped with some adaptability and robustness.