Multi-Target Joint Detection; Tracking and Classification Based on Marginal GLMB Filter and Belief Function Theory

Abstract

This paper proposes a new solution to multi-target joint detection, tracking and classification based on labeled random finite set (RFS) and belief function theory. A class dependent multi-model marginal generalized labeled multi-Bernoulli (MGLMB) filter is developed to analytically calculate the multi-target number, state estimates and model probabilities. In addition, a two-level classifier based on continuous transferable belief model (cTBM) is designed for target classification. To make full use of the kinematic characteristics for classification, both the dynamic modes and states are considered in the classifier, the model dependent class beliefs are computed on the continuous state feature subspace corresponding to different dynamic modes and then fused. As a result that the uncertainty about the classes is well described for decision, the classification results are more reasonable and robust. Moreover, as the estimation and classification problems are jointly solved, the tracking and classification performance are both improved. In the simulation, a scenario contains multi-target with miss detection and dense clutter is used. The performance of multi-target detection, tracking and classification is better than traditional methods based on Bayesian classifier or single model. Simulation results are illustrated to demonstrate the effectiveness and superiority of the proposed algorithm.

1. Introduction

Multiple target joint detection, tracking and classification is a critical problem in radar system, this problem consists of three subproblems: estimate the number of the targets, estimate their kinematic states and determine their classes. These three subproblems are usually coupled: tracking provides the kinematic features to distinguish the target type; according to the target class, appropriate dynamic models can be chosen for accurate tracking; besides, the detection of the target is the prerequisite of accurate multi-target tracking and classification [1,2,3]. Therefore, multi-target detection, tracking and classification need to be solved jointly.

The traditional joint detection, tracking and classification methods are usually developed based on the Bayesian theory and density estimation in the case of only the position measurements are available. The class probabilities and class dependent multi-target density are calculated based on the Bayesian framework. Recently, the belief function theory has been proven effective in dealing with uncertain information, and many credal models are proposed for target classification and identification, especially the transferable belief model (TBM) proposed by Smets [4,5,6]. Various joint tracking and classification algorithms are also developed based on the TBM. These methods deal with identification of objects as members of predefined classification categories. When only the position measurements are available, the target classes are mainly judged according to the kinematic features, and the proposed solution usually contains the tracker and the classifier. In [7], the dynamic behavior is described using a set of maneuver modes with different acceleration quantity according to the priori. The dynamic state estimates and model probabilities are first calculated using a bank of Kalman filters in parallel, then the beliefs and plausibilities of dynamic behaviors are computed. According to the relationship between dynamic behaviors and the target classes, the classification results are finally obtained. In [8], the Kalman filter is derived within the TBM framework, and the target joint tracking and classification problem is solved within the unified belief theoretical framework. In [9], TBM is employed in the wireless sensor networks, and the terrain information is used to improve the performance. In [10], particle filter is introduced into TBM framework to solve target joint tracking and classification problem in multi-sensor scenario. In [11], a second-order uncertainty model is proposed to describe the uncertain mapping from the dynamic feature space to the target class space, and a practical method based on TBM is provided to calculate the class likelihood under a relaxed dependence assumption. However, due to the discrete nature of such set theoretic uncertain reasoning approaches based on TBM, these algorithms have difficulty in modeling continuous signals. Therefore, the dynamic features such as the state estimates can not be directly used by the classifier. As a result, the accuracy of the classification results is affected.

To overcome such inherent defects, the belief function theory on continuous frame is recently developed by Smets, and the continuous transferable belief model (cTBM) is proposed in [12,13]. In this theory, the basic belief masses allocated on a real value is generalized into belief density on the whole real axis $\mathbb{R}$. The least committed belief functions are first built according to the priori pignistic probability density. Then the basic belief assignment of the target classes is calculated, and the classification results are finally derived using belief function tools. In these articles, the cTBM is also used for target classification, and the advantages compared to the Bayesian methods is also analyzed and illustrated. In [14], the target joint tracking and classification problem with nonlinear trajectories is solved using cTBM and particle filter. As shown in the simulations, the classification results derived based on cTBM are more accurate compared with that of the method based on TBM. However, this approach is difficult to apply to n-dimensional state space ${\mathbb{R}}^n$ unless it is independent between different dimensions [13]. Unfortunately, this is usually not satisfied during the dynamic process. In [15], the belief functions are built on ${\mathbb{R}}^2$. In [16], a new algorithm is developed when the knowledge of sensors is an $\alpha$-stable probability density function, the plausibility functions are calculated using a single Gaussian and $\alpha$-stable model. However, the computational complexities of these methods are high, and it is difficult to extend these approaches to more complicated scenarios. Therefore, in the conventional algorithms, the cTBM based classification results are usually derived based on the velocity or acceleration. Actually, the dynamic characteristics are not fully utilized.

The multi-target joint detection, tracking and classification problem in cluttered environment is much more complex, especially with target number and observation uncertainty. Inaccurate multi-target state estimates and model probabilities calculated under the condition of association error may lead to incorrect classification results. In [17], the multi-target joint tracking and classification problem is solved based on the global nearest neighbor (GNN) approach and TBM. As shown in the simulations, when there exists association error in tracking process, the classification performance is deteriorated. Recently, the random finite set (RFS) theory is developed, and many multi-target tracker are proposed, such as probability hypothesis density (PHD) filter, cardinalized PHD (CPHD) filter and cardinality balanced multi-target multi-Bernoulli (CMeMBer) filter [18,19]. These RFS based trackers are integrated approaches for multi-target joint detection and tracking, and provide the approximated multi-target density with association uncertainty. Compared to traditional approaches [20,21,22,23], the multi-target joint detection, tracking and classification problem is also solved using multi-model PHD/CPHD [1,24,25,26,27,28] and CMeMBer filter [29]. However, due to track information of the RFS based filters can not be obtained directly, these algorithms only calculate the class-dependent multi-target density without the explicit classification results for each target. In [30], the concept of labeled RFS is proposed and the conjugate multi-target state distribution is established. Several multi-target state filters are also derived, such as the generalized labeled multi-Bernoulli (GLMB) filter [31], which provides analytical solutions for multi-target tracking and produce track-valued target state estimates. As demonstrated in the simulations, the performance of the GLMB filter is much better than that of the tracker based on unlabeled RFS. In [32], the target birth intensity are adaptively modeled and introduced to the multi-Bernoulli filter to improve the estimation performance, while the filtering efficiency are maintained. However, the amount of calculation of the GLMB filter increases exponentially with time [33]. In [34], the marginal GLMB (MGLMB) density is proposed to exactly match the posterior multi-target density and the cardinality distribution, this approximate multi-target density is propagated during the estimation process and the MGLMB filter is developed. The performance of MGLMB filter is close to that of the GLMB filter, but the calculation complexity is largely reduced. In [35,36,37], multi-target detection, tracking and classification problems have also been solved using the GLMB filter, these approaches take advantages of the GLMB filter, and explicit class probabilities and state estimates of each target are produced. However, the estimation and classification results are still calculated within Bayesian theory framework.

In this paper, a joint solution is proposed to multi-target detection, tracking and classification based on labeled RFS and belief function theory in the case of only the position measurements are available. A class dependent multi-model MGLMB filter is developed to calculate the multi-target cardinality, states estimates and model probabilities, and a two-level classifier is designed based on the cTBM to identify the target classes using the kinematic data. According to different dynamic models, the class beliefs are computed on corresponding continuous dynamic feature subspace given the state estimates. Then the model-dependent class beliefs are fused to obtain the final classification results. The proposed algorithm has an analytical form, explicit track-valued state estimates and class probabilities of each target are obtained. In addition, as both the target dynamic state estimates and dynamic modes are considered, the designed classifier makes full use of the dynamic characteristics to improve the classification performance. Furthermore, because the uncertainty of the classes is well described using cTBM, and the detection, tracking and classification problems are jointly solved, both the state estimation and classification performance are improved. The particle implementation of the proposed algorithm is also detailed.

This article is organized as follows. An introduction of the cTBM and the MGLMB filter is presented in Section 2. Section 3 proposes a multi-target joint detection, tracking and classification algorithm. The simulation results are illustrated in Section 4. Finally, conclusions are summarized in Section 5.

2. Theory Foundation

2.1. Classification in the TBM Framework

The transferable belief model aims to represent quantified beliefs based on the belief functions. This model works on two levels: (1) The credal level, in which the beliefs are quantified. (2) The pignistic level, in which the decisions are made [4].

In [8], a joint tracking and classification (JTC) algorithm is proposed in TBM framework. Compared to the probability theory, the likelihood of an hypothesis is equated to the conditional plausibility of the observation given the hypothesis within the TBM. Assume that the $B=\left\{b_i\right\},i=1,\cdots ,n$ is the set of target behaviors, and $Z_t$ is the measurements, then $p{l}_i=pl(Z_t|b_i)$ indicates the measurement likelihoods that matched to behavior $b_i$. Consider a vacuous a priori on B, the posterior belief can be computed using the General Bayesian Theorem (GBT).

$$\begin{array}{c}\hfill m^B(b|Z_t)=\prod _{i:b_i\in b}p{l}_i\prod _{j:b_j\notin b}(1-p{l}_j),\forall b\subset B\end{array}$$

(1)

Then the class beliefs can be derived according to the relation between the class set C and behavior set B as

$$\begin{array}{c}\hfill m^C=\overline{M}·m^B\end{array}$$

(2)

where $m^B$ represents the vector of the basic belief assignments (BBAs), whose elements are corresponding to that in the power set of B, and similarly for $m^C$. Therefore, within the TBM framework, the beliefs allocated to the elements of the class focal set, which is the power set of the classes, and the explicit class probability is calculated until the decision is made. On contrary, the Bayesian classifier computes the explicit class probabilities based on the measurement likelihood at each time.

2.2. Belief Functions on R and Least Committed Belief Density

In [12], Smets extended the belief theory on continuous frames and proposed the continuous transferable belief model (cTBM). Compared with TBM, cTBM operates within continuous domain. The basic belief masses generalized into basic belief densities (bbds), and the belief functions can be computed by the integrations of density functions on the specific space.

Consider the construction of the belief functions on the real space $\mathbb{R}$, and a nonempty interval on the real axis $\mathbb{R}$ denoted as $[a,b]\subset \mathbb{R},a<b$. We assume that masses are only allocated to closed intervals, and the function $m:\mathcal{A}\to R$ is then a basic belief assignment, where $\mathcal{A}$ is the focal set consists of the closed intervals $A\subset \mathbb{R}$. As illustrated in Figure 1, each interval $[a,b]$ corresponds to a point K lies in the triangle, and the point $K=(a,b)$ inside the triangle ${\mathcal{T}}_{[0,1]}$ uniquely defines the interval $[a,b]\subset {\mathcal{T}}_{[0,1]}$. The belief of each point is equal to that of the corresponding interval, $P\{(x,y)=(a,b)\}=m^A([a,b])$. Consequently, the belief, plausibility and commonality of an arbitrary interval $[a,b]$ defined to be the sum of the masses of the intervals $[x,y]$ satisfies $\{x\ge a$ AND $y\le b\}$, $\{a\le x\le b$ OR $a\le y\le b\}$ and $\{x\le a$ AND $y\ge b\}$, respectively.

The commonality denotes the mass that can be assigned on any interval, within, straddling, or outside $[a,b]$. The pignistic probability is the result of mapping a belief measure to a probability measure. For the singletons within the triangle space, it can be calculated as

$$\begin{array}{c}\hfill Betf(s)=\sum _{s\in A\subset [0,1]}\frac{m(A)}{|a^{*}-a_{*}\left|\right|1-m(\varnothing )|}\end{array}$$

(3)

where $a_{*}=inf\{a:a\in A\}$ and $a^{*}=sup\{a:a\in A\}$, and $|a^{*}-a_{*}|$ is the interval length. Assume that $f^{T_{[0,1]}}$ is the basic belief density function, when the number of the focal sets is infinite, this summation is replaced by the integration

$$\begin{array}{c}\hfill Betf(a)=\underset{ϵ\to 0}{lim}{\int }_{x=0}^{x=a}{\int }_{y=a+ϵ}^{y=1}\frac{f^{T_{[0,1]}}(x,y)}{y-x}dxdy\end{array}$$

(4)

As a result that there are many bbds which share the same pignistic transformation in Equation (2), the belief density function that maximizes the commonality is defined as the least committed (LC) belief density. As given in [12], the focal interval sets of this density can be represented by a line on the triangle space, which has two properties: (1) For unimodal densities, the line starts at $(\mu ,\mu )$, $\mu =argma{x}_{a}etf(a)$ and $p{l}^T([\mu ,\mu ])=1$. (2) For all symmetrical pignistic densities $Betf$, the line is straight and centered at $\mu$

$$\begin{array}{c}\hfill y=-(x-2\mu )-\infty <x<\mu \end{array}$$

(5)

The LC belief density $\phi (a)$ of interval A is

$$\begin{array}{c}\hfill \phi (a)=-(a-\overline{a})\frac{Betf(a)}{da}\end{array}$$

(6)

where $\overline{a}$ is defined by the property that: $Betf(\overline{a})=Betf(a)$. Note that $\overline{a}$ is a function of a. The plausibility function $pl(x)$ can be computed by the integration of the basic belief density, and the limits indicate the focal intervals have non-empty intersection with x, with the property that $x\le a\le \infty$

$$\begin{array}{c}\hfill pl(x)=(x-\overline{x})Betf(x)+{\int }_x^{\infty }(1-\frac{d\overline{a}}{da})Betf(a)da\end{array}$$

(7)

Suppose that the pignistic density is Gaussian $Betf(x)=N(x;\mu ,\sigma )$, the LC belief density and corresponding plausibility $pl(x)$ are

$$\begin{array}{ccc}\hfill \kappa (y)& =& \frac{2y^2}{\sqrt{2\pi }}{e}^{-y^2/2}\hfill \end{array}$$

(8)

$$\begin{array}{ccc}\hfill pl(y)& =& \frac{2y}{\sqrt{2\pi }}{e}^{-y^2/2}+erfc(y/\sqrt{2})\hfill \end{array}$$

(9)

where $erfc(s)=(2/\sqrt{\pi }){\int }_s^{inf}{e}^{-t^2}dt$.

To solve the JTC problem in the cTBM frame, beliefs transfer when new information is received. Let $\Theta =\{{\theta }_1,\cdots ,{\theta }_n\}$ is the frame of the target classes, the posterior conditional basic belief assignment $m^{\Theta }(A|Z)$ of $x\in X$ can be calculated using the priori belief $be{l}_{\Theta }$ and the plausibility function $p{l}_{\Theta }(·|x)$ given $x\in X$ based on the GBT given in (1). Then the a posteriori conditional belief function over $\theta \in A$ can be recursively calculated using $m(A|Z)$ and the priori belief $m_{k-1}(A)$ based on the Dempster’s rule.

$$\begin{array}{c}\hfill m_k(A)=m(A|Z_k)\otimes m_{k-1}(A)\end{array}$$

(10)

The main difference between TBM and cTBM is that, in TBM the least committed plausibility is computed for the behavior, while in cTBM, the plausibility can be computed using the estimates. In the multi-target tracking scenario, the dynamic states are continuous variables, therefore, the cTBM is more effective than TBM.

2.3. Marginal Generalized Labeled Multi-Bernoulli Filter

In [30], Vo et al. introduced the notion of the labeled RFS and constructed conjugate prior. Suppose that the state vector x in the space $\mathbb{X}$ is augmented with a unique label $\ell \in \mathbb{L}$, where $\mathbb{L}$ is a discrete label space, and $\mathbf{X}$ represents the labeled RFS. Let $\mathcal{L}:\mathbb{L}\times \mathbb{X}\to \mathbb{L}$ be the projection $\mathcal{L}((x,\ell ))=\ell$, $\mathcal{L}(\mathbf{X})$ is then the label set of $\mathbf{X}$. The distinct label indicator $\Delta (\mathbf{X})={\delta }_{\left|\mathbf{X}\right|}(\mathcal{L}(\mathbf{X}))$ ensures the distinctness of the labels of $\mathbf{X}$. All the finite subsets of $\mathbb{L}$ are denoted by $\mathcal{F}(\mathbb{L})$. In a realization of the marginal generalized labeled multi-target state X, the labels of two state vectors must be distinct. The distribution of a marginal GLMB RFS is

$$\begin{array}{c}\hfill \pi (\mathbf{X})=\Delta (\mathbf{X})\sum _{I\in \mathcal{F}(\mathbb{L})}{\omega }^{(I)}{\delta }_I(\mathcal{L}(\mathbf{X})){\left[p^{(I)}\right]}^{\mathbf{X}}\end{array}$$

(11)

where $I\in \mathcal{F}(\mathbb{L})$ denotes the set of track labels, ${\omega }^{(I)}$ is the corresponding weight. $p(·,\ell )$ is the state probability density of track ℓ.

As a result that the multi-target conjugate prior constructed by introducing the notion of labeled RFSs is closed under the Chapman–Kolmogorov equation, the marginal GLMB density can be propagated within the Bayesian framework. An analytic solution for the multi-target state estimation is then provided, called marginal GLMB filter [31]. The marginal GLMB filter consists of the following two steps

1. Prediction: suppose that the multi-target prior is a GLMB density of form (12), the predicted multi-target density is then also a GLMB RFS with state space $\mathbb{X}$ and label space ${\mathbb{L}}_{+}=\mathbb{L}\cup \mathbb{B}$ ($\mathbb{L}\cap \mathbb{B}=\varnothing$), where $\mathbb{L}$ and $\mathbb{B}$ are the label spaces of the surviving and birth targets. Use the standard inner product notation $\langle f,g\rangle =\int f(x)g(x)dx$, the predicted multi-target density can be given by:

$$\begin{array}{c}\hfill {\pi }_{+}({\mathbf{X}}_{+})=\Delta ({\mathbf{X}}_{+})\sum _{I\in \mathcal{F}({\mathbb{L}}_{+})}{\omega }_{+}^{(I)}{\delta }_I(\mathcal{L}({\mathbf{X}}_{+})){\left[p_{+}^{(I)}\right]}^{{\mathbf{X}}_{+}}\end{array}$$

(12)

where

$$\begin{array}{cc}\hfill {\omega }_{+}^{(I)}& ={\omega }_B(I\cap \mathbb{B}){\omega }_s^{(I)}(I\cap \mathbb{L})\hfill \end{array}$$

(13)

$$\begin{array}{cc}\hfill p_{+}^{(I)}(x,\ell )& =1_{\mathbb{L}}(\ell )p_s^{(I)}(x,\ell )+(1-1_{\mathbb{L}}(\ell ))p_b(x,\ell )\hfill \end{array}$$

(14)

$$\begin{array}{cc}\hfill p_s^{(I)}(x,\ell )& =\frac{\langle p_s(·,\ell )f(x|·,\ell ),p^{(I)}(·,\ell )\rangle }{{\eta }_s^{(I)}(\ell )}\hfill \end{array}$$

(15)

$$\begin{array}{cc}\hfill {\eta }_s^{(I)}(\ell )& =\int \langle p_s(·,\ell )f(x|·,\ell ),p^{(I)}(·,\ell )\rangle dx\hfill \end{array}$$

(16)

$$\begin{array}{cc}\hfill {\omega }_s^{(I)}(L)& ={\left[{\eta }_s^{(I)}\right]}^L\sum _{I\subset \mathbb{L}}{1}_I(L){\left[q_s^{(I)}\right]}^{I-L}{\omega }^{(I)}\hfill \end{array}$$

(17)

$$\begin{array}{cc}\hfill q_s^{(I)}(\ell )& =\langle q_s(·,\ell ),p^{(I)}(·,\ell )\rangle \hfill \end{array}$$

(18)

For the predicted multi-target density given a predicted label set L, the weight ${\omega }_{+}^{(I)}$ is the product of the weight of the new birth and surviving target labels. The surviving target density is predicted using the transition function $f(x|·,\ell )$.

2. Update: if the multi-target prediction density is a GLMB has the form as (11), the multi-target posterior density is then also a GLMB and has the following form:

$$\begin{array}{c}\hfill \pi (\mathbf{X}|Z)=\Delta (\mathbf{X})\sum _{I\in \mathcal{F}(\mathbb{L})}\sum _{\theta \in \Theta }{\omega }^{(I,\theta )}(Z){\delta }_I(\mathcal{L}(\mathbf{X}))\times {\left[p^{(I,\theta )}(·|Z)\right]}^{\mathbf{X}}\end{array}$$

where $\Theta$ is the space of mappings $\theta :\mathbb{L}\to \{0,1,\cdots ,|Z\left|\right\}$, such that $\theta (i)=\theta (i^{\prime })>0$ implies $i=i^{\prime }$, and

$$\begin{array}{cc}\hfill {\omega }^{(I,\theta )}(Z)& \propto {\left[{\eta }_Z^{(I,\theta )}\right]}^I\hfill \end{array}$$

(19)

$$\begin{array}{cc}\hfill p^{(I,\theta )}(x,\ell |Z)& =\frac{p^{(I)}(x,\ell ){\psi }_Z(x,\ell ;\theta )}{{\eta }_Z^{(I,\theta )}(\ell )}\hfill \end{array}$$

(20)

$$\begin{array}{cc}\hfill {\eta }_Z^{(I,\theta )}(\ell )& =\langle p^{(I)}(·,\ell ),{\psi }_Z(·,\ell ;\theta )\rangle \hfill \\ \hfill {\psi }_Z(x,\ell ;\theta )& ={\delta }_0(\theta (\ell ))(1-p_d(x,\ell ))\hfill \end{array}$$

(21)

$$\begin{array}{cc}\hfill & +(1-{\delta }_0(\theta (\ell )))\frac{p_d(x,\ell )g(z_{\theta (\ell )}|x,\ell )}{\kappa (z_{(\theta (\ell ))})}\hfill \end{array}$$

(22)

In the update step, the posterior target density $p^{(I,\theta )}(x,\ell |Z)$ given labeled ℓ is explicitly calculated using the predicted multi-target density via Bayes rule with likelihood function ${\psi }_Z$. The updated weight ${\omega }^{(I,\theta )}(Z)$ is proportional to the prior weight scaled by the product ${[{\eta }_Z^{(I,\theta )}]}^I$ of single target normalizing constants. Here, $p_d(x,\ell )$ is the detection probability of track ℓ, $g(z|x,\ell )$ is the likelihood function for the measurement z and $\kappa (·)$ is the intensity of Poisson clutter.

Compute the marginalized GLMB density to exactly match the posterior multi-target density and cardinality distribution of (18), the approximate multi-target density is

$$\begin{array}{c}\hfill \widehat{\pi }(\mathbf{X})=\Delta (\mathbf{X})\sum _{I\in \mathcal{F}(\mathbf{L})}{\delta }_I(\mathcal{L}(\mathbf{X}))w^{(I)}{\left[p^{(I)}\right]}^{\mathbf{X}}\end{array}$$

(23)

where

$$\begin{array}{cc}\hfill w^{(I)}& =\sum _{\theta \in \Theta (I)}{w}^{(I,\theta )}\hfill \\ \hfill p^{(I)}(x,\ell )& =1_I(\ell )\frac{1}{w^{(I)}}\sum _{\theta \in \Theta (I)}{w}^{(I,\theta )}{p}^{(I,\theta )}(x,\ell )\hfill \end{array}$$

(24)

As given in (23), a principled GLMB $\widehat{\pi }(\mathbf{X})$ is constructed approximation to the posterior density $\pi (\mathbf{X})$, which results in a marginalization over the association histories. As a result that the number of components propagated in the Bayesian recursion is drastically reduced, the computational complexity is largely reduced.

3. Multi-Target Joint Detection, Tracking and Classification Algorithm

This section first presents the mathematical formulation of the problem, then proposes the multi-target joint detection, tracking and classification algorithm based on the labeled RFS and cTBM. The particle implementation of the proposed algorithm is also detailed.

3.1. Problem Formulation

Assume that the class of each target is a time-invariant attribute that takes values from a discrete set $\mathcal{C}=\{c_1,c_2,\cdots ,c_n\}$, and the class-dependent dynamic model set is ${\mathcal{M}}_c=\{o_1,o_2,\cdots ,o_m\}$. These models transfer given an underlying Markov process with class dependent model transition probability $f(o|o^{\prime },c)$. The target dynamic and measurement equations at time k are

$$\begin{array}{ccc}\hfill x_k& =& f_{k|k-1,o}(·|x_{k-1})+w_k\hfill \end{array}$$

(25)

$$\begin{array}{ccc}\hfill z_k& =& h_k(·|x_k)+v_k\hfill \end{array}$$

(26)

where $f_{k|k-1,o}(·)$ is the model-dependent dynamic equation, and $h_k$ is the observation function of the targets. $f_{k|k-1,o}(·)$ and $h_k(·)$ are possibly nonlinear, and $w_k$ and $v_k$ are uncorrelated Gaussian process and measurement noise.

In multi-target scenario, each target appears and disappears randomly with the birth probability $p_{b,k}$ and the survive probability $p_{s,k}$. At time k, $X_k=\{x_{k,1},x_{k,2},\cdots ,x_{k,n}\}$ is the multi-target state set with time varying target numbers, and $Z_k=\{z_{k,1},z_{k,2},\cdots ,z_{k,m}\}$ is the observation set which consists of measurements from targets and clutter with the detection probability $p_{d,k}(x_k)$ and clutter density $c_k$.

3.2. Multi-Target Joint Detection, Tracking and Classification Algorithm Based on Labeled RFS and cTBM

The proposed algorithm contains the tracker and classifier. Starting with prior multi-target density, the class dependent multi-target posterior density is first calculated using the multi-model marginal GLMB filter. Then, the classification results are derived using the two-level classifier based on cTBM. In the classifier, the model dependent class basic belief assignments (BBAs) are computed using the state estimates, then the beliefs are fused to derive the final classification results. The algorithm diagram is shown in Figure 2. The particle implementation is also provided in this section.

A. Class dependent multi-model MGLMB filter

(1) Prediction: Assume at time $k-1$, the class dependent posterior multi-target density is

$$\begin{array}{c}\hfill {\pi }_{k-1}(\mathbf{X})=\Delta (\mathbf{X})\sum _{I\in \mathcal{L}(\mathbf{X})}\sum _{c\in \mathbb{C}}{\omega }_{k-1}^{(I)}{\delta }_I(\mathcal{L}(\mathbf{X})){[p_{k-1,\ell }^{(I)}(·,o|c)]}^X\end{array}$$

(27)

where ${[p_{k-1,\ell }^{(I)}(·,o^{\prime }|c)]}^X$ is the model dependent multi-target target density given the classes. The density of each target ℓ can be further expressed as $p_{k-1,\ell }(x^{\prime },o^{\prime }|c)$. At time k, the predicted multi-target density is

$$\begin{array}{c}\hfill {\pi }_{k+}({\mathbf{X}}_{+})=\Delta ({\mathbf{X}}_{+})\sum _{I\in \mathcal{L}({\mathbf{X}}_{+})}\sum _{c\in \mathbb{C}}{\omega }_{k+}^{(I)}{\delta }_I(\mathcal{L}(\mathbf{X})){[p_{k+,\ell }^{(I)}(·,o|c)]}^{X_{+}}\end{array}$$

(28)

where

$$\begin{array}{cc}\hfill {\omega }_{k+}^{(I)}& ={\omega }_B(I\cap \mathbb{B}){\omega }_s^{(I)}(I\cap \mathbb{L})\hfill \end{array}$$

(29)

$$\begin{array}{cc}\hfill p_{k+,\ell }^{(I)}(x,o|c)& =\sum _{o\in {\mathcal{M}}_c}{p}_{k+}^{(I)}(o|c)\frac{\langle p_s^{(I)}(x^{\prime })f(x|x^{\prime },o^{\prime },c),p_{k-1,\ell }^{(I)}(x^{\prime }|o^{\prime },c)\rangle }{{\eta }_s^{(I)}(\ell )}\hfill \end{array}$$

(30)

$$\begin{array}{cc}\hfill {\eta }_s^{(I)}(\ell )& =\sum _{o^{\prime }\in {\mathcal{M}}_c}{p}_{k-1,\ell }^{(I)}(o^{\prime }|c)\int p_{s,\ell }(x^{\prime })p_{k-1,\ell }^{(I)}(x^{\prime }|o^{\prime },c)d{x}^{\prime }\hfill \end{array}$$

(31)

$$\begin{array}{cc}\hfill p_{k+,\ell }^{(I)}(o|c)& =\sum _{o^{\prime }\in {\mathcal{M}}_c}p(o|o^{\prime },c)p_{k-1,\ell }^{(I)}(o^{\prime }|c)\hfill \end{array}$$

(32)

In these equations, $\mathbb{L}$ and $\mathbb{B}$ denote the label spaces of the surviving targets and the new birth targets, respectively, $f(x|x^{\prime },o^{\prime },c)$ is the model dependent dynamic equation, $p(o|o^{\prime },c)$ is the class dependent model transition probability and $p_{s,\ell }$ is the surviving probability of track ℓ.

(2) Update: Assume that the predicted multi-target density is represented as

$$\begin{array}{c}\hfill {\pi }_{k+}({\mathbf{X}}_{+})=\Delta ({\mathbf{X}}_{+})\sum _{I\in \mathcal{L}({\mathbf{X}}_{+})}\sum _{c\in \mathbb{C}}{\omega }_{k+}^{(I)}{\delta }_I(\mathcal{L}({\mathbf{X}}_{+})){[p_{k+,\ell }^{(I)}(·,o|c)]}^{X_{+}}\end{array}$$

(33)

when the measurement set Z is received, the multi-target posterior density is

$$\begin{array}{c}\hfill {\pi }_k(\mathbf{X})=\Delta (\mathbf{X})\sum _{\xi \in \Xi }\sum _{c\in \mathbb{C}}{\omega }_k^{(\xi )}{\delta }_I(\mathcal{L}(\mathbf{X})){[p_{k,\ell }^{(\xi )}(·,o|c)]}^X\end{array}$$

(34)

where $\xi \in \Xi$ represents $(I,\theta )\in \mathcal{F}({\mathbb{L}}_{+})\times \Theta$, $\Theta$ is the space of mappings $\theta :\mathbb{L}\to \{0,1,\cdots ,|Z\left|\right\}$, such that $\theta (i)=\theta (i^{\prime })>0$ implies $i=i^{\prime }$, and

$$\begin{array}{cc}\hfill {\omega }_{c,k}^{(\xi )}(Z)& \propto {\omega }_{c,k+}^{(I)}{[{\eta }_Z^{(\xi )}]}^I\hfill \end{array}$$

(35)

$$\begin{array}{cc}\hfill p_{k,\ell }^{(\xi )}(x|o,c)& =\frac{p_{k+,\ell }^{(I)}(x|o,c){\psi }_Z^{(\xi )}(x|o,c)}{{\eta }_Z^{(\xi )}(\ell |o,c)}\hfill \end{array}$$

(36)

$$\begin{array}{cc}\hfill {\eta }_Z^{(\xi )}(\ell |o,c)& =\langle p_{k+,\ell }^{(I)}(x|o,c),{\psi }_Z^{(\xi )}(x|o,c)\rangle \hfill \end{array}$$

(37)

$$\begin{array}{cc}\hfill {\psi }_Z^{(\xi )}(x|o,c)& ={\delta }_0(\theta (\ell ))q_d(x|o,c)\hfill \end{array}$$

(38)

$$\begin{array}{cc}\hfill & +(1-{\delta }_0(\theta (\ell )))\frac{p_d(x)g(z_{\theta (\ell )}|x)}{\kappa (z_{\theta (\ell )})}\hfill \end{array}$$

(39)

$$\begin{array}{cc}\hfill p_{k,\ell }^{(\xi )}(o|c)& =\frac{p_{k+,\ell }^{(I)}(o|c){\eta }_Z^{(\xi )}(\ell |o,c)}{{\eta }_Z^{(\xi )}(\ell |c)}\hfill \end{array}$$

(40)

$$\begin{array}{cc}\hfill {\eta }_Z^{(\xi )}(\ell |c)& =\sum _{o\in {\mathcal{M}}_c}{p}_{k+,\ell }^{(I)}(o|c){\eta }_Z^{(\xi )}(\ell |o,c)\hfill \end{array}$$

(41)

In these equations, ${\delta }_0(\theta (\ell ))$ indicates whether there is a measurement associated with the target, $g(z_{\theta (\ell )}|x)$ is the likelihood function and $p_d$ is the detection probability of the target. ${[p_{k,\ell }^{(\xi )}(·,o|c)]}^X$ is the product of multiple target densities $p_{k,\ell }^{(\xi )}(·,o|c)$ given predicted labels I and association map $\theta$. The class dependent marginal GLMB density that matches the class dependent posterior multi-target density and the cardinality distribution is

$$\begin{array}{c}\hfill \widehat{\pi }(\mathbf{X})=\Delta (\mathbf{X})\sum _{I\in \mathcal{F}(\mathbf{L})}\sum _{c\in \mathbb{C}}{\omega }_k^{(I)}{\delta }_I(\mathcal{L}(\mathbf{X})){[p_{k,\ell }^{(I)}(·,o|c)]}^X\end{array}$$

(42)

where

$$\begin{array}{cc}\hfill {\omega }_k^{(I)}& =\sum _{\theta \in \Theta }{\omega }_k^{(\xi )}\hfill \end{array}$$

(43)

$$\begin{array}{cc}\hfill p_{k,\ell }^{(I)}(x,o|c)& =1_I(\ell )\frac{1}{{\omega }_k^{(I)}}\sum _{\theta \in \Theta }{\omega }_k^{(\xi )}{p}_{k,\ell }^{(\xi )}(x,o|c)\hfill \end{array}$$

(44)

and the target number is

$$\begin{array}{c}\hfill N=\sum _{I\in \mathcal{F}(\mathbf{L})}n\times {\omega }_k^{(I)}\end{array}$$

(45)

where n is the number of the targets in label set $\mathbf{L}$.

B. Classification based on cTBM

The dynamic features are associated with both dynamic states and motion modes. For example, the target shows a higher maneuverability when executing a sharp turn than fling straight at the same speed; the cruising speed of fighters may exceed sonic velocity, which is much faster than most normal planes. Obviously, the classification based on either dynamic states or motion modes is partial. Therefore, a two-level classifier is designed here, and both of these two factors are considered. The proposed classifier consists of the following components.

(1) Classification on the state feature subspace

As a result that the dynamic states of different dimensions are usually considered as joint, such as the speed and acceleration, the calculation of the beliefs based on cTBM in the framework of a one-dimensional continuous state feature space is difficult to be directly extended to multi-dimensional feature space. Assume that $x_{k,o,\ell }$ is the posterior state estimate of track $\ell \in I$ given the model o. In the proposed algorithm, one-dimensional state feature subspace is firstly chosen corresponding to the dynamic models, and the conditional LC plausibility $pl(x_{k,o,\ell }|c_i)$ is calculated using the state sub-vector ${\tilde{x}}_{k,o,\ell }\in \mathbb{R}$. Then the beliefs are fused to derive the final classification results. Suppose that $Betf({\tilde{x}}_{k,o,\ell }|c_i,o_j)$ is the prior pignistic density function of target class $c_i$ and model $o_j$, then the LC plausibility can be computed as

$$\begin{array}{ccc}\hfill pl({\tilde{x}}_{k,o,\ell }|c_i,o_j)& =& ({\tilde{x}}_{k,o,\ell }-\varphi ({\tilde{x}}_{k,o,\ell }))Betf({\tilde{x}}_{k,o,\ell }|c_i,o_j)\hfill \end{array}$$

(46)

$$\begin{array}{cc}+& {\int }_{{\tilde{x}}_{k,o,\ell }}^{\infty }(1-\frac{d\varphi (a)}{da})Betf(a|c_i,o_j)da\hfill \end{array}$$

(47)

where $\varphi ({\tilde{x}}_{k,o,\ell })$ satisfies

$$\begin{array}{c}\hfill Betf(\varphi ({\tilde{x}}_{k,o,\ell })|c_i,o_j)=Betf({\tilde{x}}_{k,o,\ell }|c_i,o_j)\end{array}$$

(48)

and the state sub-vector ${\tilde{x}}_{k,o,\ell }$ is selectable according to different modes. For example, velocity or acceleration are chosen for classifying corresponding to CV or CA models, respectively. Using the Generalized Bayesian Theorem (GBT), the model dependent BBAs of the classes are

$$\begin{array}{c}\hfill m(C|{\tilde{x}}_{k,o,\ell },o_j)=\prod _{c_i\in C}pl({\tilde{x}}_{k,o,\ell }|c_i,o_j)\times \prod _{c_i\in \overline{C}}[1-pl({\tilde{x}}_{k,o,\ell }|c_i,o_j)]\end{array}$$

(49)

(2) Classification with multi-model

Let $(\mathbb{C},\mathbb{X},\mathbb{O})$ indicates the credibility space, where $\mathbb{X}$ and $\mathbb{O}$ are the state and model frames, respectively, while the beliefs allocated to the elements of $\mathbb{C}$. Combine the model dependent beliefs $m(C|{\tilde{x}}_{k,o,\ell },o)$ calculated before, the final class BBAs of target ℓ is

$$\begin{array}{c}\hfill m(C|x_{k,\ell })=\sum _{o_j\in {\mathcal{M}}_c}m(C|{\tilde{x}}_{k,o,\ell },o_j)m(o_j)\end{array}$$

(50)

where $m(o_j)$ is the confidence allocated on the models, which is equal to the model probability [38]. This equation can also be represented using the matrix equation $m^{\mathcal{C}}=\overline{M}{m}^o$, where $m^{\mathcal{C}}=[m^{c_1},m^{c_2},\cdots ,m^{c_n}]$ is the vector of BBAs, $\mathcal{C}$ is the power set of the class, and $m^o=[m^{o_1},m^{o_2},\cdots ,m^{o_m}]$ is the vector of model belief masses. The dimension of these two vectors are not need to be equal, this conforms the situation that the dynamic models and the classes are not in one-to-one correspondence. Each column of the matrix $\overline{M}$ is the model dependent BBAs of class focal given the selectable state sub-vector ${\tilde{x}}_{k,o,\ell }$.

(3) Time recursion using Generalized Bayesian Theorem

Combine the conditional beliefs with prior information using (9), the posterior class beliefs $m_k(C|x_{k,o,\ell })$ at time k can be calculated. Then, the pignistic class probabilities are

$$\begin{array}{c}\hfill Bet{P}_k\left\{c_i\right|x_{k,o,\ell }\}=\sum _{c_i\in C}\frac{1}{\left|C\right|}\frac{m_k(C|x_{k,o,\ell })}{1-m_k(\varnothing |x_{k,o,\ell })}\end{array}$$

(51)

The proposed MGLMB-cTBM algorithm for multi-target joint detection, tracking and classification based on labeled RFS and belief function theory is summarized as follows:

Algorithm 1: MGLMB-cTBM algorithm for multi-target joint detection, tracking and classification

Initial the prior conditional pignistic density function of the dynamic states. (1) Predict the multi-target density ${\pi }_{k+}({\mathbf{X}}_{+})$, and model dependent density $p_{k+,\ell }^{(I)}(x,o|c)$ and model probability $p_{k+,\ell }^{(I)}(o|c)$ of each target using (28)–(32). (2) Update the multi-target density ${\pi }_k(\mathbf{X})$, model dependent density $p_{k,\ell }^{(\xi )}(x|o,c)$ and model probability $p_{k,\ell }^{(\xi )}(o|c)$ using (34)–(41). (3) Compute the marginal GLMB density $\widehat{\pi }(\mathbf{X})$ using (42)–(44). (4) For different dynamic models, chose the state estimate sub-vector ${\tilde{x}}_{k,o,\ell }\in \mathbb{R}$ to calculate the conditional LC plausibility $pl({\tilde{x}}_{k,o,\ell }|c_i,o_j)$, and then compute the model dependent beliefs $m(C|{\tilde{x}}_{k,o,\ell },o_j)$ using (46)–(48). (5) Combine the model dependent beliefs to compute the final BBAs of the class focal $m(C|x_{k,\ell })$ as (49). (6) Merge the beliefs with the priori using GBT, and then go to step 1.

Analysis and discussion: 1. In this paper, an analytical algorithm is proposed to multi-target joint detection, tracking and classification in radar system. In this application, the target is embodied in the form of points. For the point targets, all the information about the target can be represented using the state and category. Therefore, both the observations and the dynamic information can be directly denoted using the measurement and state vectors. The posterior state and class probabilities can be analytically updated, and the classification results can be explicitly derived based on the dynamic states and the prior distribution of the class pignistic probability obtained directly from statistical data. Therefore, the derived results are Bayesian optimal. Compared to the traditional methods in [28,36], the dependence between estimation and classification is considered in this paper. Multi-target density is predicted using the class dependent multi-model, and the classifier computes the target class according to the both the dynamic models and estimates. The relation between multi-target class and state estimation is reflected in both the prediction and update process of the proposed algorithm. As given in (36)–(42), the mapping weights are updated based on the class dependent multi-model likelihood, therefore, the target existing probability (detection) is derived related to the target class in (45). As given in (43)–(45), the target density is the sum of weighed density updated given the target classes, therefore, the multi-target tracking result is also related to target classes. Therefore, according to the target class, appropriate models are used for improve the detection and tracking performance. Furthermore, more accurate estimation results lead to the improvement of the classification and overall performance.

2. In the cTBM based classifier, the classification results are explicitly derived based on the dynamic state estimates and the prior distribution of the class pignistic probability obtained directly from statistical data. Therefore, the classification results provided by the classifier are Bayesian optimal. During the solution process, the LC plausibilities of target classes are first computed, then the BBAs of the class focal sets are derived. The target class focal set is the power set of the target classes. The number of the focal elements in the set is $2^n$, where n is the number of possible class in the frame of discernment. The beliefs allocated to the elements of the class focal set, which is the power set of the classes. For example, when the two categories $c1$ and $c2$ can not be completely distinguished, the belief mass is assigned to the elements $\left\{c1c2\right\}$. The belief assignments are combine with the prior beliefs using the GBT, and the explicit class probability is calculated until the decision is made. On contrary, the Bayesian classifier computes the class probabilities based on the measurement likelihood at each time. Moreover, compared with the TBM based methods, cTBM based classifier can derive the classification results directly using the dynamic estimates. When there is no effective information contained in the dynamic model (the target moves straight with the same velocity all the time), the class can still be judged according to the dynamic estimates. In addition, the models are selected according to the dynamic modes. By contrast, for the TBM, the classification result is derived based on the dynamic modes, which is actually an interval in the state space, such as the accelerate $a\in [-g,+g]$. Therefore, the uncertainty about the class is described more precisely and the cTBM based classifier is better than the traditional approaches. Furthermore, as the number of model is largely reduced, the computational complexity is reduced and the model competition is avoided.

3. The computation complexity of the proposed MGLMB-cTBM algorithm is generally higher than that of the standard GLMB filter. Assume that $P=max|\mathcal{L}(X)|$ is the maximum number of the targets, and $M=\left|Z\right|$ is the maximum number of the measurements, as given in [33], the computational complexity of GLMB filter is $\mathcal{O}(P^{2}M)$, i.e., quadratic in the number of hypothesized labels and linear in the number of measurements. The computational complexity of marginal GLMB filter is the same as that of the GLMB filter in the worst case. In practical applications, for the classifier of the proposed MGLMB-cTBM algorithm, the integral calculation in (46) can be calculated previously based on the prior pignistic probability density, and the belief assignments given the estimates can be quickly obtained by looking up the table. Therefore, the main difference of the computational complexity between the MGLMB-cTBM and MGLMB filter lies in the estimation process given different classes and models, which involves predicting and updating the target state according to the class dependent multi-model. Assume that C and O are the maximum number of the target class focal sets and models, the computational complexity of MGLMB-cTBM is $\mathcal{O}({(COP)}^{2}M)$.

4. With the application of optoelectronic technology and the improvement of radar resolution, the target morphological information can also be obtained, such targets are treated as the image targets and extended targets, respectively. In [39,40,41], the graph neural network (GNN) is used for multiple image targets tracking. The graph nodes are composed of all the targets in all frames. The attributes of the nodes are composed of apparent features and geometric features (position and shape) obtained through training, and the distance of the nodes are defined using Euclidean distance, etc. There are connections between nodes across frames, and the weights of the edges are setting based on the defined distance metric. The association and tracking results are derived through recursively updating the weights of the edges and the attributes of the nodes. In [42,43], convolutional neural networks (CNN) and long short-term memory (LSTM) are used to obtain the kinematic and morphological characteristics of the target, and the dynamic states and association relationships of the targets are calculated using the GNN. In [44], the importance of learned reID features for multi-object tracking is shown. These proposed GNN-based methods can also be used for multiple extended targets tracking. Moreover, in the scenario when the attribute measurements are available, such as the signal features, the belief of the attribute measurements can be fused with the beliefs calculated using the dynamic estimates based on the D-S evidence theory. Relevant works can be done in the future.

3.3. The Particle Implementation

(1) Prediction: As given in (28), the predicted multi-target density ${\pi }_{k+}({\mathbf{X}}_{+})$ consists of the model dependent density of each target. Using the particle description, the density $p_{k+,\ell }^{(I)}(x,o|c)$ of track ℓ can be denoted as ${\{x_{k+,o,\ell }^{(n)},{\omega }_{k+,o,\ell }^{(n)}\}}_{n=1}^{N_{k+,o,\ell }}$, where $x_{k+,o,\ell }$ and ${\omega }_{k+,o,\ell }$ are the predicted state and weight that calculated using the state transition matrix $F_o$ and surviving probability $p_s^{\ell }$

$$\begin{array}{ccc}\hfill x_{k+,o,\ell }^{(n)}& =& F_o{x}_{k-1,o,\ell }^{(n)}\hfill \end{array}$$

(52)

$$\begin{array}{ccc}\hfill {\omega }_{k+,o,\ell }^{(n)}& =& p_s^{\ell }{\omega }_{k-1,o,\ell }^{(n)}\hfill \end{array}$$

(53)

The predicted density of new birth target ℓ can be indicated using particles ${\{x_{b,o,\ell }^{(n)},{\omega }_{b,o,\ell }^{(n)}\}}_{n=1}^{N_{b,o,\ell }}$, where the state $x_{b,o,\ell }^{(n)}$ and weight ${\omega }_{b,o,\ell }^{(n)}$ conform the prior.

(2) Update: The posterior density $p_{k,\ell }^{(\xi )}(x|o,c)$ of target ℓ can be indicated using particles ${\{x_{k,o,\ell }^{(n)},{\omega }_{k,o,\ell }^{(n)}\}}_{n=1}^{N_{k,o,\ell }}$, where the weights

$$\begin{array}{c}\hfill {\omega }_{k,o,\ell }^{(n)}={\omega }_{k+,o,\ell }^{(n)}\left({\delta }_0(\theta (\ell ))(1-p_d)+(1-{\delta }_0(\theta (\ell )))\frac{p_{d}q(x_{k_{+},o,\ell }^{(n)})\delta \left(x_k^{(n)}-x_{k+}^{(n)},o_k^{(n)}-o_{k+}^{(n)}\right)}{\kappa (z_{\theta (\ell )})+{\sum }_{o\in {\mathcal{M}}_c}{\sum }_{n=1}^{N_{k+,o,\ell }}q(x_{k_{+},o,\ell }^{(n)})}\right)\end{array}$$

(54)

$$\begin{array}{c}\hfill q(x_{k_{+},o,\ell }^{(n)})=\mathcal{N}(z_{\theta (\ell )};H_k{x}_k^{(n)},R_k)\end{array}$$

(55)

In these equations, the term $x_{k,o,\ell }^{(n)}$ and ${\omega }_{k,o,\ell }^{(n)}$ are the posterior state particles and associated weights, ${\delta }_0(\theta (\ell ))$ indicates whether there is a measurement associated with the target, $q(x_{k+,o,\ell }^{(n)})$ is the likelihood function, and $p_d$ is the detection probability of the target. The dynamic model probabilities are $p_{c,k}(o)={\sum }_{n=1}^{N_{k+,o,\ell }}{\omega }_{c,k,o,\ell }^{(n)}$.

(3) Resample: To avoid particle impoverishment and prevent exponential growth of the particle number, a resample procedure is performed, and then the derived particles have equal weights

$$\begin{array}{c}\hfill {\overline{\omega }}_{k,o,\ell }^{(n)}=\frac{{\omega }_{k,o,\ell }^{(n)}}{{\sum }_{o\in {\mathcal{M}}_c}{\sum }_{n=1}^{N_{k+,o,\ell }}{\omega }_{k,o,\ell }^{(n)}}\end{array}$$

(56)

(4) Compute the BBAs: For each particle $\{x_{k,o,\ell }^{(n)},{\omega }_{k,o,\ell }^{(n)}\}$, the state sub-vector ${\tilde{x}}_{k,o,\ell }$ is selectable according to different modes $o_j$. Suppose that $Betf({\tilde{x}}_{k,o,\ell }^{(n)}|c_i,o_j)$ is the prior pignistic density function of target class $c_i$ and model $o_j$. Then, the LC plausibility and BBAs can be computed using the state estimates of each particle as

$$\begin{array}{ccc}\hfill pl({\tilde{x}}_{k,o,\ell }^{(n)}|c_i,o_j)& =& ({\tilde{x}}_{k,o,\ell }^{(n)}-\varphi ({\tilde{x}}_{k,o,\ell }^{(n)}))Betf({\tilde{x}}_{k,o,\ell }^{(n)}|c_i,o_j)\hfill \end{array}$$

(57)

$$\begin{array}{cc}+& {\int }_{{\tilde{x}}_{k,o,\ell }^{(n)}}^{\infty }(1-\frac{d\varphi (a)}{da})Betf(a|c_i,o_j)da\hfill \end{array}$$

(58)

$$\begin{array}{c}\hfill m(C|{\tilde{x}}_{k,o,\ell }^{(n)},o_j)=\prod _{c_i\in C}pl({\tilde{x}}_{k,o,\ell }^{(n)}|c_i,o_j)\times \prod _{c_i\in \overline{C}}[1-pl({\tilde{x}}_{k,o,\ell }^{(n)}|c_i,o_j)]\end{array}$$

(59)

Combine all of the BBAs from each of the particles for model o using D-S theory [14], the fused model conditioned BBAs can be calculated as

$$\begin{array}{c}\hfill m_{1\oplus 2}(C|{\tilde{x}}_{k,o,\ell },o_j)=\frac{{\sum }_{B\cap C}m(C|{\tilde{x}}_{k,o,\ell }^1,o_j)m(C|{\tilde{x}}_{k,o,\ell }^2,o_j)}{1-{\sum }_{B\cap C=\varnothing }m(C|{\tilde{x}}_{k,o,\ell }^1,o_j)m(C|{\tilde{x}}_{k,o,\ell }^2,o_j)}\end{array}$$

(60)

(5) Classification with multi-model: Combine the model dependent beliefs $m(C|{\tilde{x}}_{k,o,\ell },o)$, the final class BBAs of target ℓ is

$$\begin{array}{c}\hfill m(C|x_{k,\ell })=\sum _{o_j\in {\mathcal{M}}_c}m(C|{\tilde{x}}_{k,o,\ell },o_j)m(o_j)\end{array}$$

(61)

Fuse the conditional beliefs with prior information to compute the posterior class beliefs $m_k(C|x_{k,o,\ell })$. Then, the pignistic class probabilities are

$$\begin{array}{c}\hfill Bet{P}_k\left\{c_i\right|x_{k,o,\ell }\}=\sum _{c_i\in C}\frac{1}{\left|C\right|}\frac{m_k(C|x_{k,o,\ell })}{1-m_k(\varnothing |x_{k,o,\ell })}\end{array}$$

(62)

At last, proportionally correct the weights of the particles belong to the class $c_i$ conform that ${\sum }_{o\in {\mathcal{M}}_c}{\omega }_{k,o,\ell }^{(n)}=Bet{P}_k\left\{c_i\right|x_{k,o,\ell }\}$.

The implementation can be realized using the open source tools (http://ba-tuong.vo-au.com/codes.html), and the pseudo-code of the MGLMB-cTBM algorithm is given as follows:

Algorithm 2: Pseudo-code of the proposed MGLMB-cTBM algorithm

1: function MGLMB-cTBM algorithm 2: Input ${\{x_{k+,o,\ell }^{(n)},{\omega }_{k+,o,\ell }^{(n)}\}}_{n=1}^{N_{k+,o,\ell }}$ given different classes and models, input the measurements $Z_k$. 3: Generate the k-shortest path to calculate k-best births and surviving hypotheses, predict the target state $x_{k+,o,\ell }^{(n)}$ and ${\omega }_{k+,o,\ell }^{(n)}$ using (52) and (53), add the birth particles ${\{x_{b,o,\ell }^{(n)},{\omega }_{b,o,\ell }^{(n)}\}}_{n=1}^{N_{b,o,\ell }}$. 5: for $i=1,\cdots ,|Z_k|$ 6: Compute the likelihood $q(z_i|x_{k_{+},o,\ell }^{(n)})$ for each measurement $z_i\in Z_k$ using (55). 7: end for 8: Calculate m-best assignment hypotheses/components using murty’s algorithm according to the likelihood $q(z_i|x_k^{(n)})$. 9: Update the weights ${\omega }_{k,o,\ell }^{(n)}$ using (54). 10: Resample. 11: Look up the table and obtain the LC plausibility $pl({\tilde{x}}_{k,o,\ell }^{(n)}|c_i,o_j)$ and BBAs $m(C|{\tilde{x}}_{k,o,\ell }^{(n)},o_j)$ according to the estimates $x_{k+,o,\ell }^{(n)}$ of particles. Compute the dynamic model probability $p_k(o)={\sum }_{n=1}^{N_{k+,o,\ell }}{\omega }_{k,o,\ell }^{(n)}$. 12: Combine the BBAs for model $m_{1\oplus 2}(C|{\tilde{x}}_{k,o,\ell },o_j)$ using (59). 13: Combine the model dependent beliefs $m(C|{\tilde{x}}_{k,o,\ell },o)$, and then merge the beliefs with the priori using GBT to derive the final class BBAs of target ℓ. Then go to 1. end function

4. Simulations

In this section, simulations are performed to illustrate the effectiveness of the proposed algorithm. The scenarios under consideration contain several targets form three categories: ordinary targets, medium maneuvering target and high maneuvering targets, such as commercial plane, bomber and fighter, respectively. The model set consists of the constant velocity, constant acceleration and coordinated turn models. For the commercial plane, the transition probability of the dynamic models is $0.05$, while the transition probabilities for the bomber and fighter are $0.1$ and $0.2$, respectively. The prior pignistic probability distributions of velocity, acceleration and turn rate for each category are Gaussian, i.e., $Betf\sim N(\overline{x},\sigma )$, and the parameters are given in

Table 1. Numerical values for priori pignistic class probability density.

	Commercial Plane	Bomber	Fighter
Velocity interval (km/s)	[680, 1020]	[850, 1190]	[720, 1580]
Velocity var (km/s)	120	160	250
Accelerate mean (g)	0	0	0
Accelerate var (g)	0.8	1.2	1.6
Turn rate mean (rad/s)	0	0	0
Turn rate var (rad/s)	0.1	0.2	0.4

. In Figure 3, the pignistic probability functions of these classes are illustrated using thick, normal and dashed line, respectively.

In our method, different state feature subspaces are chosen for classification according to the dynamic modes. For CV, CA and CT models, the class beliefs are calculated based on the velocity, acceleration and turn angular rate, respectively. The performance of our algorithm is compared with the traditional methods in terms of the class probability.

In the scenario, five targets are involved, two are the high maneuvering targets such as the fighter, two other are the ordinary targets such as the normal commercial planes and the last one is a medium maneuvering target such as the bomber. These targets perform different maneuvers in $(x,y)$ space. Target 1 appears at the beginning and firstly evolves with a constant velocity 870 km/h until 24 s, then executes a coordinated turn during 24 to 32 s at 0.35 rad/s. Finally, it performs a normal flight and disappears at 80 s. Target 2 appears at 8 s and disappears at the end, it firstly evolves with a low constant velocity at 850 km/h, then turns during 24 to 30 s with the turning angle rate 0.2 rad/s. Last, it speeds up to 1200 km/h within 8 s and escapes. Targets 3 and 4 appear from 16 s, and perform the uniform speed flight at 780 km/h throughout the process. Target 5 appears at 20 s and performs the uniform speed flight at 1080 km/h. The scenario is illustrated in Figure 4. The measurements of the tracks are drawn using black ‘*’, and the clutter are drawn using blue ‘*’.

In the simulation, the equation of the ith dynamic model is

$$\begin{array}{c}\hfill x_k=F_{k,i}{x}_{k-1}+w_{k,i}\end{array}$$

(63)

where $F_{k,i}$ is the model-dependent state transition matrix, and $w_{k,i}$ is Gaussian noise with covariance $Q_{k,i}$. The constant velocity (CV) model has the following parameters

$$\begin{array}{cc}\hfill F_{k,1}& =diag\left(\left[\begin{array}{ccc}1& T& 0\\ 0& 1& 0\\ 0& 0& 0\end{array}\right],\left[\begin{array}{ccc}1& T& 0\\ 0& 1& 0\\ 0& 0& 0\end{array}\right],1\right)\hfill \end{array}$$

(64)

$$\begin{array}{cc}\hfill Q_{k,1}& =diag\left(\left[\begin{array}{ccc}{T}^2& T& 0\\ T& 1& 0\\ 0& 0& 0\end{array}\right]{\sigma }_v^2,\left[\begin{array}{ccc}{T}^2& T& 0\\ T& 1& 0\\ 0& 0& 0\end{array}\right]{\sigma }_v^2,0\right)\hfill \end{array}$$

(65)

where ${\sigma }_v$ is the process noise with covariance ${\sigma }_v^2=10m^2/s^2$. The parameters of CA model are

$$\begin{array}{cc}\hfill F_{k,2}& =diag\left(\left[\begin{array}{ccc}1& T& \frac{1}{2}{T}^2\\ 0& 1& T\\ 0& 0& 1\end{array}\right],\left[\begin{array}{ccc}1& T& \frac{1}{2}{T}^2\\ 0& 1& T\\ 0& 0& 1\end{array}\right],1\right)\hfill \end{array}$$

(66)

$$\begin{array}{cc}\hfill Q_{k,2}& =diag\left(\left[\begin{array}{ccc}\frac{1}{4}{T}^4& \frac{1}{2}{T}^3& \frac{1}{2}{T}^2\\ \frac{1}{2}{T}^3& T^2& T\\ \frac{1}{2}{T}^2& T& 1\end{array}\right]{\sigma }_a^2,\left[\begin{array}{ccc}\frac{1}{4}{T}^4& \frac{1}{2}{T}^3& \frac{1}{2}{T}^2\\ \frac{1}{2}{T}^3& T^2& T\\ \frac{1}{2}{T}^2& T& 1\end{array}\right]{\sigma }_a^2,0\right)\hfill \end{array}$$

(67)

where ${\sigma }_a$ is the process noise with covariance ${\sigma }_a^2=10$ m${}^2/$s${}^4$. The parameters of CT model are

$$\begin{array}{cc}\hfill F_{k,3}& =\left[\begin{array}{ccccccc}1& \frac{sin(\omega T)}{\omega }& 0& 0& -\frac{1-cos(\omega T)}{\omega }& 0& 0\\ 0& cos(\omega T)& 0& 0& -sin(\omega T)& 0& 0\\ 0& 0& 1& 0& 0& 0& 0\\ 0& \frac{1-cos(\omega T)}{\omega }& 0& 1& \frac{sin(\omega T)}{\omega }& 0& 0\\ 0& sin(\omega T)& 0& 0& cos(\omega T)& 0& 0\\ 0& 0& 0& 0& 0& 1& 0\\ 0& 0& 0& 0& 0& 0& 1\end{array}\right]\hfill \end{array}$$

(68)

$$\begin{array}{cc}\hfill Q_{k,3}& =diag\left(\left[\begin{array}{cc}\frac{T^3{l}_1}{3}& \frac{T^2{l}_1}{2}\\ \frac{T^2{l}_1}{2}& T{l}_1\end{array}\right],0,\left[\begin{array}{cc}\frac{T^3{l}_1}{3}& \frac{T^2{l}_1}{2}\\ \frac{T^2{l}_1}{2}& T{l}_1\end{array}\right],0,T{l}_2\right)\hfill \end{array}$$

(69)

where $l_1$ and $l_2$ are the process noise with covariance $l_1={10}^{-1}ra{d}^2/s^{-4}$ and $l_2={10}^{-4}ra{d}^2/s^{-4}$, respectively.

The sensor receives range and bearing measurements

$$\begin{array}{c}\hfill z_k=\left[\begin{array}{c}{l}_k\\ {\theta }_k\end{array}\right]+v_k\end{array}$$

(70)

where $l_k$ and ${\theta }_k$ are the true range and bearing measurement of a target that can be given by

$$\begin{array}{ccc}\hfill l_k& =& \sqrt{{(p_x-s_x)}^2+{(p_y-s_y)}^2}\hfill \end{array}$$

(71)

$$\begin{array}{ccc}\hfill {\theta }_k& =& arctan\left[\frac{p_x-s_x}{p_y-s_y}\right]\hfill \end{array}$$

(72)

The position of the sensor $[p_x,p_y]$ is located at $[1000,-3000]$, and $v_k$ is the Gaussian measurement noise with the covariance $R_k=diag({\sigma }_r^2,{\sigma }_{\theta }^2)$, where ${\sigma }_r=5m$ and ${\sigma }_{\theta }=(\pi /180)rad$. The target detection probability is $p_d=0.92$, there are on average 35 clutters per scan within the multi-target moving area (clutter density is $2.08\times {10}^{-7}$), and the sample time is 2 s. The target survival probability is $p_s=0.98$, and the target birth probability is $p_b=0.02$. The density of the new birth target is $b_k=N(x;m_b;Q_b)$, where the parameters $m_b$ is equal to the position of the first measurement of each target, assume that the measurement in Cartesian coordinates is $[x,y]$, the initial state of the birth targets is $m_b={[x,0,0,y,0,0,0]}^{\prime }$, while the state covariance is $Q_b=diag(100,{10}^4,10,100,{10}^4,10,0.1)$, respectively.

As shown in Figure 5a,b, take the targets 1 and 5 as example, the dynamic modes are correctly identified by the multi-model MGLMB filter in the case of dense clutter and miss detections. The model dependent class beliefs are calculated given corresponding dynamic state characteristics and then fused. The final classification results are illustrated in Figure 5c,d. For target 1, the pignistic class probability of commercial plane is the largest at first, the reason is that it performs a normal flight, and the classification result is mainly derived on basis of the velocity. When it turns, the pignistic probability of fighter rapidly increases because only the this kind of the target can do such sharp maneuver with high turning angular rate. For target 5, the object flies at a constant speed from beginning to the end, therefore, the classification results are derived using the velocity, the pignistic probabilities of fighter and bomber are approximately both equal to $1/2$, while the commercial plane is excluded by the velocity. As a result that the kinematic characteristics are not very distinguishable, the target class cannot be effectively distinguished only from the speed, therefore the classification results still contain much uncertainty about the fighter and bomber.

The comparison of the classification results for one target of the proposed MGLMB-cTBM algorithm and the Bayesian classifier is shown in Figure 6. Take the target 1 as example, before the target does a maneuver, the probability of normal commercial plane is larger using the Bayesian approach. In contrast, the classification results of the proposed MGLMB-cTBM algorithm still contain much uncertainties. When the target turns, the probability of the target being the fighter is the highest in both algorithms. After the target completes the maneuver, the probability of commercial plane obtained by the Bayesian classifier significantly increases again. The reason is that, the specific classification results are produces by the Bayesian classifier at each moment, and the prior probability of the commercial plane is larger under such speed. On contrary, for the classifier based on cTBM, the uncertainty of the target class is indicated using the confidence assigned to the focal element set $C=\{c_1,c_2,c_{1,2}\cdots \}$, and the reliabilities are translated into pignistic probabilities only when the decision is made. Therefore, when the evidences are insufficient, the difference among the pignistic class probabilities of different categories is not obvious. This result reflects the reasonability of the proposed MGLMB-cTBM algorithm.

In Figure 7, the superiority of the two level classifier is shown. Take the target 2 as example, as illustrated in the Figure 7a, the class probabilities of three classes are represented by the thick, normal and dash line, respectively. Before the target accelerates, the pignistic probability of commercial plane is larger because the classification results are derived mainly based on the velocity. After the medium maneuver performed during 24–32 s, the pignistic probabilities of bomber and fighter increase promptly. However, there still exists uncertainty about fighter and bomber, because all types of aircraft can do this maneuver except the commercial plane. Finally, the target class is determined on basis of the high velocity. The classification results of the method proposed in [7] are shown in Figure 7b, in which the target class is identified only based on the acceleration. It can be seen that, because the target makes medium maneuvering from 24 to 32s, the target class can be preliminary judged according to the acceleration, but no effective classification result is derived when the acceleration is 0 during the cruise section of flight. The kinematic characteristics are not fully utilized, and the acceleration is insufficient for distinguishing of the bomber and fighter absolutely. In contrast, our method identifies the motion mode firstly and classifies using appropriate dynamic characteristics, the target is finally identified according to the high speed. The classification performance is improved.

In Figure 8, the comparison of the overall detection, tracking and classification performance of 50 Monte Carol runs is illustrated. The proposed MGLMB-cTBM algorithm is compared with the MGLMB-Bayesian method in terms of the cardinality estimates, class dependent optimal subpattern assignment (OSPA) like distance for tracks, and average error class probability (the error between the classification results and the truth). In MGLMB-Bayesian method, the target class is introduced into conventional multi-model MGLMB filter, the posterior class probabilities and class dependent multi-target density are calculated based on the observation likelihood within Bayesian framework. As shown in Figure 8c, because the classification results derived within belief function framework is more reasonable and robust, the classification results of the cTBM based classifier is better. In Figure 8a,b, the detection and overall tracking performance of our proposed method is also better. The reason is that, according to more accurate classification results, appropriate dynamic modes are chosen during the estimation process, and the overall performance is improved. In Figure 8c The time scale represents the moving time of the targets. The proposed algorithm has an iterative implementation form, the motion state and class probability of each target are recursively calculated at each moment, and the number of iteration is actually equal to the time scan. The values of average error class probability change over time because the conditional beliefs are calculated at the current moment based on the dynamic state estimates, and then combined with the prior information using generalized Bayesian theorem. As more dynamic information of the target obtained, the class are more definite with the time recursion, and the values of average error class probability decreased over time.

To further evaluate the overall performance of the proposed algorithm, the algorithm works under 100 different scenarios that are randomly generated. Each scenario contains 5 targets moving within 90 s. The real class, appearing time, initial dynamic state and position of these targets are the same as in scenario 1. For the fighter, the beginning time of maneuver is random, and the probability of maneuvering is 0.02 at each moment during 10–80 s. The covariance of the Gaussian measurement noise $R_k=diag(200;5\times {10}^{-3})$, and there are average 50 clutters per scan within the multi-target moving area, both larger than scenario 1.

The proposed MGLMB-cTBM algorithm is compared with MGLMB-Bayesian and PHD-TBM algorithms [28] in terms of the cardinality estimates, overall tracking performance, and average error class probability. As illustrated in Figure 9a, although the cardinality estimates of all the algorithms converges to the truth, the estimates of PHD-TBM is worse within several scans after the target appears, the reason is that the MGLMB is an analytical multi-target tracker, and the dynamic state estimates is more accurate than PHD filter. In Figure 9c, the classification result of TBM-PHD is also worse because the results are derived only based on the dynamic modes, while the estimates are directly used for classification in cTBM based classifier. Moreover, both the classification and state estimation of MGLMB-cTBM algorithm is better than MGLMB-Bayesian method. Actually, the traditional method based on Bayesian framework is optimal in the usual sense, the performance of the proposed algorithm is better because it takes advantages of the belief function theory, the classification results derived within belief function framework is more reasonable and robust, as the target class is considered in state estimation, appropriate models are then used for multi-target tracking. As a result that the estimation and classification problems are jointly solved, more accurate classification and estimation results are both helpful to improve the overall performance.

The proposed MGLMB-cTBM algorithm is realized using Matlab and the open source tools. In the simulation, the density of each target is represented using 1000 particles. The operating environment of the program is Windows 10, i5 5200 Dual-core CPU 2.2 GHz, 8 GB RAM. The computational time of each step of MGLMB-cTBM, MGLMB-Bayesian and PHD-TBM methods is 15.7 s, 14.5 s and 0.095 s, respectively. The computational time of these algorithms is given in

Table 2. Average computational time.

MGLMB-cTBM	MGLMB-Bayesian	PHD-TBM
15.7 s	14.5 s	0.095 s

. As shown in the table, the computational time of PHD-TBM is far less than the other two methods, because the computational complexity of the PHD-based tracker is much lower than the MGLMB filter. As the target state estimates of MGLMB-cTBM need to be calculated given different classes and models, in addition, compared to the MGLMB-Bayesian filter, the beliefs of the particles need to be fused. Therefore, the computational time of MGLMB-cTBM algorithm is the largest. As illustrated in Figure 9, the estimation and classification results of MGLMB-cTBM are the best, therefore, the increase in the amount of the computation leads to an improvement in the performance. Actually, the Gaussian mixture implementation can be used under linear Gaussian condition, and the computational time of MGLMB-cTBM algorithm can be further reduced.

5. Conclusions

This paper presents a new solution to multi-target joint detection, tracking and classification with target number and observation uncertainty based on the marginal GLMB filter and cTBM. Explicit state estimates and pignistic class probabilities of each target are produced by the proposed algorithm, and the particle implementation is also given. For multi-target tracking, a class dependent multi-model MGLMB filter is developed to calculate the target existing probabilities, dynamic model probabilities and model-dependent target state estimates. For target classification, a two-level classifier is designed, the class beliefs are first computed on corresponding state feature subspace according to different models, and then fused to derive the final results. Numerical examples are also performed to demonstrate the effectiveness and robustness of the proposed algorithm. As both the dynamic modes and the state estimates are considered by the classifier, the kinematic characteristics are fully utilized to improve the classification performance. Moreover, because the explicit class piginistic probability is only computed when the decision is made, the uncertainty of the class is well described using the belief functions during the recursion, and the classification results are more reasonable. Actually, the designed classifier extracts more information from the measurement and kinematic states to adaptively improve the tracking performance by choosing the proper motion modes, etc. In addition, because the estimation and classification problems are jointly solved, the performance of the detection, tracking and classification are further improved compared with the traditional methods.