**1. Introduction**

While single-object tracking algorithms have been studied extensively for more than half a century, multi-object tracking is currently a trending topic in signal processing society due to its extensive applications. The challenges of the multi-object tracking problem arise in the context of miss-detection, false alarms, object thinning, and appearing processes. To tackle these problems, several frameworks have been put forward in the literature such as the Joint Probabilistic Data Association (JPDA) [1], multiple hypotheses tracking [2], and recently, Random Finite Sets (RFS) [2]. In particular, RFS forms the mathematical basis of many modern multi-object filters such as Probability Hypothesis Density (PHD) filter [3–7], cardinalized PHD (CPHD) filter [8–10], multi-Bernoulli filter [11,12], the Generalized Labeled Multi-Bernoulli (GLMB) filter [13–19], and its approximation the Labeled Multi-Bernoulli (LMB) filter [20,21]. In many applications, tracking algorithms rely on the standard point measurements to update the object states; in contrast, TBD [22–25] is an alternative approach that bypasses the detection module to directly exploit the observed spatial data. This technique is introduced under the RFS framework in Reference [26] with the development of the so-called separable likelihood model and, recently, in a hybrid (combination of standard observation and separable observation models) approach in Reference [27]. In terms of system modelling, in many multi-object tracking scenarios, it is sufficient to consider object thinning and appearing processes via survivals, deaths, and instantaneous birth models. However, in many practical applications, new objects are also generated from a set or a subset of existing objects. In the context of RFS-based filtering techniques, such spawning models have been proposed for CPHD filter in References [28,29] and for GLMB filter in Reference [30]. Because these spawning models correctly reflect the physical state of the systems with spawning objects, the accuracy of the estimate is improved.

The early works on practical smoothing algorithms for single-object tracking were introduced by Bryson and Frazier [31]; by Rauch, Tung, and Striebel [32]; and subsequently by Fraser and Potter [33]. Later on, many alternative smoothing algorithms for a nonlinear dynamic model were proposed in References [34–39]. Recently, the closed-form solution for the Gaussian Mixture (GM) forward-backward smoother was derived in Reference [3]. Furthermore, the smoother for the multi-sensor tracking problem is addressed in Reference [40], while the smoothing solution for maneuvering-object tracking is presented in Reference [41]. In Reference [42], a method for joint tracking smoothing of object trajectory based on function fitting is also proposed. In a multi-object tracking context, several smoothing techniques have been put forward in the literature despite the challenge of the large smoothing state space. In particular, smoothing for PHD filter is introduced in References [43–45], while smoothing for CPHD filter and Multi-Bernoulli filter are given respectively in References [46,47]. Multiple objects can also be tracked with a fixed-lag smoother via Interacting Multiple Model (IMM) in Reference [48]. Closed-form solution for forward-backward smoothing based on GLMB RFS is introduced in Reference [49], and recently, multi-scan smoothing technique was proposed in Reference [50] with an efficient implementation based on Gibbs sampling, which can easily handle 100 scans. This is an unprecedented advance over traditional multi-scan solutions, which can only handle about 10 scans.

The labeled RFS approach has several theoretical and practical advantages over unlabeled approaches. The first is that labeled RFS filters can provide trajectory estimates naturally without heuristics, whereas this is not possible with unlabeled RFS filters; see Reference [51]. The second is that labeled RFS can provide ancestry information in a principled manner, whereas unlabeled RFS does not have the mechanism to do this (even with smoothing) [30]. The third is that labeled RFS admits analytical solutions such as the GLMB densities that are still valid RFS densities after any truncation, whereas unlabeled RFS cannot; see [51]. The fourth is that the truncation error (or error bound) for labeled RFS, such as GLMB, is available analytically, whereas this is not available for unlabeled RFS [13]. Consequently, truncation-based unlabeled RFS algorithms are heuristics [51]. Numerically, labeled RFS filters such as the GLMB have been demonstrated to be scalable in the number of objects [52], number of scans [50], and number of sensors [53]. Hence, the GLMB is a versatile class of models for multi-sensor multi-object problems.

In this paper, we introduce a tracker based on the GLMB filters and a modification of the multi-scan estimator proposed in Reference [50]. After the forward GLMB filtering stage, a pre-smooth stage is implemented to eliminate short-term tracks, which are usually initiated by false births or spawns. The threshold to prune these tracks varies depending on the tracking scenario. Subsequently, a multi-scan estimator which consists of a standard single-object filter and an Rauch–Tung–Striebel (RTS) smoother is applied on each estimated trajectory to produce smoothed estimates. As the proposed multi-scan estimator operates only on the estimated trajectories, the complexity is much lower than the full smoothing solution proposed in Reference [50]. Especially, this proposed tracker can completely eliminate track fragmentation as the multi-scan estimator estimates the entire trajectories but not single-scan multi-object states as in standard GLMB filters [13,15,16]. We demonstrate the application of the proposed tracker on both a standard measurement model and a TBD measurement model as well as tracking scenario with object spawning.

The structure of this paper is as follows. In Section 2, we provide background information on labeled RFSs, the multi-object transition kernel, the observation models, and the single-object RTS smoother for linear and nonlinear dynamic models. In Section 3, we propose the tracker based on the GLMB filters and the multi-scan estimator. In Section 4, we first show the experimental results for tracking with the standard observation model in linear and nonlinear tracking scenarios. We then show the tracking results of the proposed algorithm with a hybrid observation model. Finally, we demonstrate the performance of the algorithm on tracking biological cells in an image sequence where spawning process occurs.
