Pedestrian Trajectory Prediction Based on Motion Pattern De-Perturbation Strategy

Abstract

Pedestrian trajectory prediction is extremely challenging due to the complex social attributes of pedestrians. Introducing latent vectors to model trajectory multimodality has become the latest mainstream solution idea. However, previous approaches have overlooked the effects of redundancy that arise from the introduction of latent vectors. Additionally, they often fail to consider the inherent interference of pedestrians with no trajectory history during model training. This results in the model’s inability to fully utilize the training data. Therefore, we propose a two-stage motion pattern de-perturbation strategy, which is a plug-and-play approach that introduces optimization features to model the redundancy effect caused by latent vectors, which helps to eliminate the redundancy effects in the trajectory prediction phase. We also propose loss masks to reduce the interference of invalid data during training to accurately model pedestrian motion patterns with strong physical interpretability. Our comparative experiments on the publicly available ETH and UCY pedestrian trajectory datasets, as well as the Stanford UAV dataset, show that our optimization strategy achieves better pedestrian trajectory prediction accuracies than a range of state-of-the-art baseline models; in particular, our optimization strategy effectively absorbs the training data to assist the baseline models in achieving optimal modeling accuracy.

1. Introduction

The task of pedestrian trajectory prediction is to predict the future trajectories of pedestrians by modeling their movement patterns on the basis of historical trajectories in real-world scenarios. This task is used in various automated scenarios, such as traffic, service, and video surveillance. Accurately predicting the future trajectories of pedestrians can help avoid collisions between driverless devices and pedestrians. Additionally, it can identify abnormal behaviours and assess crowd density in public places, particularly in dense urban scenarios.

For trajectory prediction tasks, it is still challenging to accurately model the movement patterns of pedestrians. This is due to the complex social interactions involved and the fact that pedestrian movements are easily influenced by surrounding pedestrians and the environment.

In previous methods [1,2,3,4,5,6], historical pedestrian behavioral characteristics, such as trajectory characteristics and interaction characteristics, were mainly derived from historical trajectory information and subsequently combined with the introduction of latent vectors (tensors of random numbers obeying a standard normal distribution) to predict the multimodal future trajectories of the pedestrians. These approaches focus on improving pedestrian interactions and modeling pedestrian multimodality. However, they consistently ignore the fact that the latent vectors are composed of random numbers unrelated to the historical behavioral characteristics of pedestrians. We believe that using latent vectors to model the randomness of pedestrian motion has a redundancy effect on the accurate modeling of pedestrian motion patterns. This redundancy effect will be explored through the experiments in

Table A1. Redundancy effect exploration experiment (STGAT w/o LZ means no latent vectors; STGAT with OF means that our optimization features were added).

Method	Performance (ADE/FDE)
	ETH	HOTEL	ZARA1	ZARA2	UNIV	AVG
STGAT	0.90/1.76	0.48/0.99	0.43/0.92	0.33/0.72	0.55/1.18	0.54/1.11
STGAT w/o LZ	0.84/1.74	0.50/1.05	0.41/0.88	0.32/0.71	0.52/1.12	0.52/1.10
STGAT with OF	0.89/1.75	0.47/0.95	0.42/0.90	0.32/0.70	0.53/1.13	0.53/1.09

of Appendix A.1. Furthermore, these methods do not take into account the inherent interference that exists during model training, which we define as the perturbations of pedestrians without historical trajectories for model training. As a result, these methods are ineffective in optimizing the performance of the constructed models.

In Figure 1, panel (a) shows a schematic diagram for constructing pedestrian motion patterns, modeling the time domain of historical trajectories to obtain trajectory features, modeling the spatial domain of historical trajectories to obtain interaction features, and sampling latent vectors from a normal distribution to characterize the multimodality of pedestrian motion. Most of the methods are based on this idea to establish the motion patterns of pedestrians. Latent vectors are introduced to predict multimodal trajectories (i.e., multiple possibilities of trajectories), but the latent vectors themselves are random numbers that are completely unrelated to the trajectory features and interaction features; thus, the introduction of latent vectors will inevitably interfere with the expression of the trajectory features and interaction features, thus introducing unnecessary redundancy effects in the subsequent trajectory prediction. Moreover, there are still inherent interference factors in network training for learning pedestrian motion patterns, i.e., the presence of pedestrians with no history of trajectories. In panel (b) of Figure 1, the model training phase calculates the predicted loss for pedestrians with and without historical track information. In panel (c), the predicted loss is calculated only for pedestrians with historical track information. It should be noted that in panel (c), the predicted trajectory error for pedestrians with no historical trajectories is simply disregarded in the loss calculation. This does not affect the interaction between pedestrians in the model of panel (a). The right half of panel (b) demonstrates that pedestrians without historical trajectory information have multiple possible future trajectories that are entirely random. As a result, there is no correlation between their future and historical trajectories.

To solve these problems and model more accurate and reliable pedestrian motion patterns, we propose a two-stage motion pattern de-perturbation strategy. Stage 1—Constructing the pedestrian motion pattern, the baseline model utilizes latent vectors to introduce randomness. Inspired by the counterfactual analysis paper [7], they use counterfactual features to replace trajectory features to eliminate training and deployment environment biases. We replace trajectory and interaction features with optimization features to eliminate redundant effects of introducing latent vectors. Stage 2—The training model iteration phase is a new loss mask design for shielding pedestrians with no history of trajectories to more accurately model pedestrian motion patterns and accurately capture the correspondence between learned historical trajectories and future trajectories. Notably, we do not interfere with the interaction between pedestrians with no history track and other pedestrians. Our proposed two-stage motion pattern de-perturbation strategy is a plug-and-play module. It can be applied to any benchmark method for pedestrian trajectory prediction that introduces latent vectors. We conducted experiments on three baseline approaches, namely, STGAT [3], which is based on an RNN (Recurrent Neural Network) and a GAT (Graph Attention Network); SGAN [2], which is based on an RNN and a GAN (Generative Adversarial Network); and SocialVAE [6], which is based on an RNN and a VAE (Variational Auto-Encoder).

In summary, this paper contributes the following:

• The introduction of latent vectors inevitably introduces redundancy effects; therefore, we propose the use of optimization features to replace trajectory features and interaction features to eliminate these redundant effects.
• Pedestrians without historical trajectories during model training can interfere with the accurate iteration of pedestrian motion patterns; thus, we propose loss masks to eliminate this interference to reduce the uncertainty of the training process.
• Our method, as a migratable module, can effectively eliminate the interference factor of pedestrian motion pattern modeling in two stages, maximizing the performance of the baseline models.

2. Related Work

Expert-based models generally set the relevant rules manually, and some approaches base their motion planning on dynamics equations and obstacles in the scene [8,9], while others are based on heuristic methods that use human-formulated motion functions to avoid collisions [10,11]. There are also methods [12,13] that use pedestrian interaction simulation software, such as Legion, to simulate complex interactions between pedestrians and analyze group motion behaviors more effectively.

In recent years, many methods have been applied to learning pedestrian movement patterns from data. The data-driven models mainly include Bayesian network-based, reinforcement learning-based, and deep learning-based models. Among them, deep learning-based models are the focus of this paper.

Bayesian networks were used in [14] to model an agent’s state and predict the agent’s intention and trajectory. Several approaches [15,16] also use deep reinforcement learning models to predict pedestrian trajectories in human–vehicle conflict scenarios.

Deep models have recently achieved good results in pedestrian trajectory prediction tasks due to the powerful characterization capabilities of deep learning.

Regarding recurrent neural networks, speech sequence recognition and machine translation [17,18] have demonstrated that RNNs and their variants, such as long short-term memory networks (LSTMs), are well suited for processing sequence information and predicting sequence data problems. Social-LSTM [1] uses the hidden states extracted by an RNN as the trajectory features of pedestrians and introduces a local “social” pooling module by considering the social attributes of pedestrians to weight the hidden states of pedestrians within a certain spatial distance to simulate pedestrian interactions in real scenes. Social-GAN [2] uses a GAN as the overall model architecture and uses an RNN to model pedestrian trajectory features in the generator but uses a global pooling module to improve the pooling mechanism based on the relative distance between pedestrians, making the generated trajectories more in line with social norms. TPHT [19] introduces a soft attention mechanism to model the interaction between pedestrians based on RNN-extracted trajectory features. Several works [6,20,21,22,23,24] further refine the extraction of pedestrian interaction features using scene information and the physical parameters of pedestrian motion (motion direction angle, shortest distance, etc.).

Graph neural networks have been widely used in the field of deep learning in recent years. Previous methods have been based on designing pooling modules to aggregate interactions between pedestrians; this pooling method aggregates only different pedestrian states based on simple features such as the spatial distance between pedestrians, which has certain limitations, while graph neural networks essentially combine graph data with neural networks to effectively aggregate information between nodes, which is highly relevant to pedestrian trajectory prediction tasks. STGAT [3] introduces GAT to design spatio-temporal graphical attention networks. RNN is used to encode pedestrian trajectories, while GAT is used to capture the spatial correlation between different pedestrian trajectories to extract social interactions between pedestrians. Social-BiGAT [25] applied a GAT-based generative adversarial network to better model pedestrian social interactions in scenarios, while adversarial training based on the GAN framework was used to model the multimodality of pedestrian trajectories. STGCNN [26] and SGCN [27] model the movement patterns of pedestrians directly and explicitly on a spatiotemporal graph, modeling interactions as graphs to replace traditional aggregation methods. Some work [28,29] combines rule-based or optimized physical models with deep learning models to improve model accuracy.

Numerous methods [1,2,3,4,5,6,20,22,25] rely on the randomness of latent vectors to create the multimodality of pedestrian trajectories. However, they often overlook the redundancy effects caused by the latent vectors being unrelated to pedestrian movement patterns. Moreover, these methods often overlook the impact of pedestrians without historical trajectories on the accurate iterative motion patterns during the training model stage. Given this, our two-stage de-disturbance strategy incorporates optimized features and loss masks to eliminate redundant disturbances. This enables the baseline models to demonstrate even better performance.

3. Methods

This section outlines the construction of a two-stage motion pattern de-perturbation strategy and its implementation in the baseline models.

Table 1. Summary of main notations.

Notation	Description
$X_i$	Past trajectory of agent $i$
$Y_i$	Future trajectory of agent $i$
$m_i$	Trajectory features of agent $i$
$g_i$	Interaction features of agent $i$
$z$	Latent vectors sampled from a Gaussian distribution
$W_d$	Weight parameter in the baseline model decoder
$\left|\right|$	Concatenation operation
$h_i$	Pedestrian motion patterns output by the baseline model decoder
${\xi }_i$	Optimization factor utilized to construct optimization features
$OF$	Optimization features
${h_i}^{’}$	Decoder output using optimization features
$W_o$	Weight parameter of the output module in the baseline model
$\widehat{Y_i}$	Final predicted trajectory obtained using optimization features
$∆x_i^{t_1},{∆y}_i^{t_1}$	Coordinate component of the displacement of agent $i$ at time $t_1$
$V_A^i$	Average speed of agent $i$ over historical observation frames
${\mathcal{L}}_B$	Loss function defined by the baseline model
${\mathcal{L}}_{Mask}^i$	Loss mask for agent $i$

provides a summary of the main notations used in this paper.

3.1. Problem Definition

The goal of pedestrian trajectory prediction is to take all pedestrian historical coordinate points at a specified time $t_1=1,2,\dots ,t_{obs}$ as input $X=X_1, X_2, \dots , X_n$ and to predict the future coordinate points $\widehat{Y}={\widehat{Y}}_1, {\widehat{Y}}_2, \dots , {\widehat{Y}}_n$ of all pedestrians at a subsequent time $t_2=t_{obs}+1, t_{obs}+2,\dots , t_{pred}$ by constructing a model. For pedestrian $i$, its input historical coordinates and true future coordinates can be defined as follows:

$$X_i=\left\{\left\{\left(x_i^{t_1},y_i^{t_1}\right)\in R^2\right\}|t_1=1,2,\dots ,t_{obs}\right\}$$

(1)

$$Y_i=\left\{\left\{\left(x_i^{t_2},y_i^{t_2}\right)\in R^2\right\}|t_2=t_{obs}+1, t_{obs}+2,\dots , t_{pred}\right\}$$

(2)

Notably, previous methods mainly used the relative position of pedestrians, i.e., the historical displacement, instead of absolute position coordinates as input information.

3.2. Method Overview

Encoder–Decoder framework: Pedestrian trajectory prediction is a sequence prediction task. Many methods use coding and decoding frameworks, with encoding and decoding modules typically designed as RNN or LSTM structures. The process can be summarized as follows:

• The encoding module encodes the input historical pedestrian trajectory to obtain the trajectory features of the pedestrians.
• The portrayal of interactions between pedestrians is generally based on factors such as their distance from each other or the risk of collision. This is performed to obtain the encoding of the interaction feature.
• The complete motion pattern encoding of the pedestrians is obtained by splicing together the trajectory features, interaction features, and introduced latent vectors.
• To decode the high-dimensional features of the predicted trajectory, input the complete motion pattern encoding of pedestrians into the decoding module.
• The predicted trajectory’s high-dimensional features are passed through the output module, typically a linear layer, to obtain the mapped future trajectory.

Overview: As shown in Figure 2, where the encoder and decoder represent the encoding and decoding modules of a baseline model, respectively, the de-perturbation strategy consists of two main stages. In the first stage, optimization features are used to replace pedestrian trajectory and interaction features in the baseline model, eliminating the redundancy effects of introducing latent vectors; in the second stage, the designed loss mask is used to optimize the model training process, eliminating the interference of irrelevant training data for loss computation.

3.3. Stage 1: Pedestrian Motion Pattern Construction Optimization

As shown in panel (a) of Figure 1, the baseline model introduces randomness of latent vectors to model the multimodality of trajectories. However, according to our experimental results in Table A1 in Appendix A.1, latent vectors cause redundant effects on the motion patterns constructed by the baseline model. Therefore, as shown in Figure 2, we propose to use optimization features to replace the pedestrian trajectory features and interaction features connected to the latent vectors, and the same are input to the decoding module of the baseline model to replicate the redundancy effect caused by the introduction of latent vectors.

For pedestrian $i$, we assume that the trajectory features and interaction features modeled by the baseline model are $m_i$ and $g_i$, respectively, so that the pedestrian motion pattern $h_i$ modeled by the baseline model can be given by the following equation:

$$h_i=Decoder\left(m_i‖g_i‖z;W_d\right)$$

(3)

where $\left|\right|$ denotes the concatenation operation and $z$ is the latent vector introduced by the baseline model to construct the multimodality of the pedestrian trajectory, i.e., to introduce randomness into the deterministic trajectory. $W_d$ is the weight parameter in the decoder module.

The additional optimization features we introduce are defined as $OF=\left(m_i‖g_i\right)\ast {\xi }_i$, where ${\xi }_i$ is the optimization factor (set to zero in our approach); moreover, we present our exploratory experiments using different forms of ${\xi }_i$ in Section 4. $OF$ is used to replace the trajectory features ($m_i$) and interaction features ($g_i$) in the pedestrian movement pattern constructed by the baseline model to replicate the redundancy effects (${h_i}^{’}$) from the introduction of latent vectors:

$$h_i^{’}=Decoder\left(OF‖z;W_d\right)$$

(4)

$h_i$ and $h_i^{’}$ are finally input to the output module of the baseline model to eliminate redundancy effects to obtain the final predicted trajectory:

$$\widehat{Y_i}=\mathrm{O}\mathrm{u}\mathrm{t}\mathrm{p}\mathrm{u}\mathrm{t}\left(h_i-h_i^{’};W_o\right)$$

(5)

where $W_o$ is the weight parameter in the output module of the baseline model and $\widehat{Y_i}$ is the predicted trajectory of the final model output.

3.4. Stage 2: Loss Mask Optimization

We first use the historical coordinates $X_i$ of pedestrian $i$ to calculate the displacement of the current time frame relative to the previous time frame:

$$∆x_i^{t_1}=x_i^{t_1}-x_i^{t_1-1},{∆y}_i^{t_1}=y_i^{t_1}-y_i^{t_1-1}$$

(6)

We calculate the average velocity ($V_A^i$) of pedestrian $i$ based on its displacement in the historical observation frames. The $V_A^i$ in the historical observation frame is described as follows:

$$V_A^i=\frac{1}{t_{obs}}{\sum }_{t_1=1}^{t_{obs}}\sqrt{{\left(∆x_i^{t_1}\right)}^2+{\left({∆y}_i^{t_1}\right)}^2}$$

(7)

For pedestrian $i$ with zero average velocity $V_A^i$, the displacement in historical time is also zero, and it is impossible to model its motion pattern to catch the relationship between historical and future trajectories. For this reason, we propose the loss mask (${\mathcal{L}}_{Mask}^i$) for training to ensure that the gradient backpropagation of loss is not used for pedestrian $i$, so that the network no longer focuses on pedestrian $i$ during training and more accurately models pedestrians with definite motion patterns. We define ${\mathcal{L}}_{Mask}^i$ below:

$${\mathcal{L}}_{Mask}^i=\left\{\begin{array}{c}ones, V_A^i\ne 0\\ zeros, V_A^i=0\end{array}\right.$$

(8)

The final loss function is defined as follows:

$$\mathcal{L}=\mathcal{L}(Y_i, \widehat{Y_i})\ast {\mathcal{L}}_{Mask}^i$$

(9)

where ${\mathcal{L}}_B$ is the loss function defined by the baseline model and the training loss mask ${\mathcal{L}}_{Mask}^i$ enables the network to target weight updates to exclude the interference of irrelevant data to more accurately map the relationship between historical and future trajectories, ensuring the accuracy of modeled pedestrian movement patterns and making the network model more effectively absorb training data and improving learning efficiency. In Section 4, we conducted experiments to investigate the model’s learning efficiency with varying dataset sizes.

4. Experimental Section

4.1. Experimental Settings

Dataset: We evaluated our method on two publicly available trajectory prediction datasets, ETH [30] and UCY [31], which contain five different real scenarios, namely, ETH, Hotel, Zara1, Zara2, and Univ. These scenarios involve different social environments and different crowd densities. The dataset was obtained by sampling data from a real scene at 0.4 s intervals, and the training idea was the same as that used by the current mainstream methods; that is, 3.2 s ($t_{obs}=8$) trajectory data were used as the historical trajectory data, and 4.8 s ($t_{pred}=12$) trajectory data were used as the real values of the predicted trajectories while maintaining a cross-validation evaluation strategy that is consistent with the baseline models [2,3,6].

Evaluation metrics: To specifically evaluate the accuracy of the predicted trajectory, we used the same evaluation metrics as those used by the baseline methods, the average displacement error (ADE) [32] and the final displacement error (FDE) [1], which are calculated as follows:

$$\mathrm{A}\mathrm{D}\mathrm{E}=\frac{\sum _{i\in N}{\sum }_{t=t_{obs}}^{t_{pred}}{‖{\widehat{Y}}_n^t-Y_n^t‖}_2}{N\ast (t_{pred}-t_{obs)}}$$

(10)

$$\mathrm{F}\mathrm{D}\mathrm{E}=\frac{\sum _{i\in N}{‖{\widehat{Y}}_i^{t=t_{pred}}-Y_i^{t=t_{pred}}‖}_2}{N}$$

(11)

Baseline models: We used three representative baseline models for trajectory prediction to evaluate our approach: (1) an SGAN based on an RNN and a GAN, (2) an STGAT based on an RNN and a GNN, and (3) a SocialVAE based on an RNN and VAE.

4.2. Experimental Evaluation

In this section, we show how the experiments demonstrated the effectiveness of the proposed two-stage motion pattern de-perturbation strategy.

Experimental evaluation of the pedestrian motion pattern de-perturbation strategy: To evaluate the effectiveness of our designed method in improving prediction accuracy, we conducted comparison experiments on the official source codes provided by STGAT [3], Social-GAN [2], and SocialVAE [6], as shown in

Table 2. Evaluation results of several advanced baseline models. Our two-stage optimization strategy improved the predictive efficacy of all three baseline methods. For both the ADE and FDE metrics, the lower the value is, the better the result. The table highlights the best results for ADE/FDE in bold.

Baseline 1	Performance (ADE/FDE)
	ETH	HOTEL	ZARA1	ZARA2	UNIV	AVG
Social LSTM [1]	1.09/2.35	0.86/1.91	0.41/0.88	0.52/1.11	0.61/1.31	0.70/1.52
SoPhie [20]	0.70/1.43	0.76/1.67	0.30/0.63	0.38/0.78	0.54/1.24	0.54/1.15
SR-LSTM [33]	0.63/1.25	0.37/0.74	0.41/0.90	0.32/0.70	0.51/1.10	0.45/0.94
STSGN [34]	0.75/1.63	0.63/1.01	0.30/0.65	0.26/0.57	0.48/1.08	0.48/0.99
MATF [35]	1.33/2.49	0.51/0.95	0.44/0.93	0.34/0.73	0.56/1.19	0.64/1.26
MATF GAN [35]	1.01/1.75	0.43/0.80	0.26/0.45	0.26/0.57	0.44/0.91	0.48/0.90
PITF [21]	0.73/1.65	0.30/0.59	0.38/0.81	0.31/0.68	0.60/1.27	0.46/1.00
Social-BiGAT [25]	0.69/1.29	0.49/1.01	0.30/0.62	0.36/0.75	0.55/1.32	0.48/1.00
STGCNN [26] STGAT [3]	0.64/1.11 0.65/1.12	0.49/0.85 0.35/0.66	0.34/0.53 0.34/0.69	0.30/0.48 0.29/0.60	0.44/0.79 0.52/1.10	0.44/0.75 0.43/0.83
STGAT *	0.80/1.42	0.37/0.70	0.33/0.66	0.29/0.61	0.55/1.17	0.47/0.91
Ours–STGAT	0.59/1.02	0.34/0.61	0.32/0.65	0.30/0.62	0.52/1.12	0.41/0.80
Baseline 2	Performance (ADE/FDE)
	ETH	HOTEL	ZARA1	ZARA2	UNIV	AVG
SGAN [2]	0.71/1.29	0.48/1.02	0.34/0.69	0.31/0.64	0.56/1.18	0.48/0.96
SGAN *	0.75/1.36	0.41/0.82	0.33/0.68	0.30/0.64	0.53/1.13	0.46/0.93
Ours–SGAN	0.64/1.15	0.39/0.75	0.33/0.67	0.29/0.62	0.53/1.15	0.44/0.87
Baseline 3	Performance (ADE/FDE)
	ETH	HOTEL	ZARA1	ZARA2	UNIV	AVG
PECNet [36]	0.54/0.87	0.18/0.24	0.22/0.39	0.17/0.30	0.35/0.60	0.29/0.48
Trajectron++ [37]	0.54/0.94	0.16/0.28	0.21/0.42	0.16/0.31	0.28/0.55	0.27/0.50
SGCN [27]	0.63/1.03	0.32/0.55	0.29/0.53	0.25/0.45	0.37/0.70	0.37/0.65
Social-Implicit [38]	0.66/1.44	0.20/0.36	0.25/0.50	0.22/0.43	0.31/0.60	0.33/0.67
SocialVAE [6]	0.49/0.77	0.15/0.24	0.19/0.37	0.15/0.28	0.25/0.47	0.25/0.43
SocialVAE *	0.50/0.85	0.15/0.23	0.20/0.37	0.16/0.29	0.25/0.48	0.25/0.44
Ours–SocialVAE	0.48/0.77	0.15/0.21	0.21/0.37	0.15/0.28	0.26/0.49	0.25/0.42

, where * in Table 2 indicates the results we directly replicated with the official source codes; for fairness of comparison, we kept the same hyperparameters as the replicated baseline model source codes. When evaluating Ours–STGAT, Ours–SGAN, and Ours–SocialVAE, we kept the same hyperparameters as the reproduced baseline model source codes, and we set the same random number seed for all subsequent experimental evaluations to make the experimental results reproducible. The results in Table 2 show that our proposed two-stage motion pattern de-perturbation strategy can improve the prediction accuracy of the different baseline models, where Ours–STGAT achieves an improvement of 0.06/0.09 on the average ADE/FDE on five different scenario datasets compared to the baseline STGAT* model, Ours–SGAN achieves an improvement of 0.02/0.06 compared to the baseline SGAN* model, and Ours–SocialVAE achieves an improvement of 0.02 on the average FDE compared to the baseline model. Notably, our strategy results in all baseline models achieving significant increases in both the ETH and HOTEL dataset scenarios, with the increases being more pronounced in the ETH scenario. We attribute this to the fact that pedestrians in motion in the ETH and HOTEL scenarios are more sparsely located; thus, pedestrian motion is more influenced by their own motion patterns. In the ETH and HOTEL datasets, our strategy removes the redundancy effects of the randomness of latent vectors on the construction of pedestrian motion patterns, while the use of loss masks shields the inherent interference of pedestrians with no history of trajectories, allowing for the maximization of baseline model performance.

Evaluating the use of different forms of ${\xi }_i$ to construct the optimization features: To determine the final form of the optimization features introduced in Section 3.2, as shown in

Table 3. The results were evaluated when using different forms of ${\xi }_i$ to construct the optimized features, where STGAT is the baseline model. The table highlights the best results for ADE/FDE in bold.

	Performance (ADE/FDE)
	ETH	HOTEL	ZARA1	ZARA2	UNIV	AVG
STGAT* (Baseline)	0.80/1.42	0.37/0.70	0.33/0.66	0.29/0.61	0.55/1.17	0.47/0.91
STGAT 1	0.59/1.02	0.34/0.61	0.32/0.65	0.30/0.62	0.52/1.12	0.41/0.80
STGAT 2	0.61/1.04	0.37/0.73	0.32/0.66	0.30/0.60	0.52/1.11	0.42/0.83
STGAT 3	0.71/1.25	0.35/0.64	0.33/0.65	0.30/0.61	0.56/1.17	0.45/0.86
STGAT 4	0.67/1.24	0.33/0.62	0.33/0.67	0.31/0.63	0.55/1.15	0.44/0.86

Note: ¹ indicates that ${\xi }_i$ used zero vectors, ² indicates that all the values in ${\xi }_i$ were 0.5, ³ indicates that ${\xi }_i$ was obtained by sampling from a uniform distribution, and ⁴ indicates that ${\xi }_i$ was obtained by sampling from a standard normal distribution.

, we designed experiments to evaluate the effect of using different forms of ${\xi }_i$ to construct the optimization features to obtain the final experimental results. The experiments involved using ${\xi }_i$ with all-zero values, ${\xi }_i$ with all-0.5 values, ${\xi }_i$ with values sampled from a uniform distribution, and ${\xi }_i$ with values sampled from a standard normal distribution. The experimental results in Table 3 show that the baseline model is improved under every form of ${\xi }_i$, which indicates that our method has some universality. Using ${\xi }_i$ with all-zero vectors achieves the best results, which is consistent with our design philosophy that the optimization features are introduced to eliminate the redundancy of latent vectors, and the use of all-zero vectors can directly reflect the influence of latent vectors on trajectory features and interaction features.

Contributions of different optimization designs: In this subsection, we discuss the design of experiments to evaluate the separate contributions of the optimization features and the loss mask, as shown in

Table 4. The separate contributions of the optimized features and loss masks were evaluated using STGAT as a baseline model. The table highlights the best results for ADE/FDE in bold.

	Performance (ADE/FDE)
	ETH	HOTEL	ZARA1	ZARA2	UNIV	AVG
STGAT* (Baseline)	0.80/1.42	0.37/0.70	0.33/0.66	0.29/0.61	0.55/1.17	0.47/0.91
STGAT w/o OF	0.77/1.33	0.36/0.67	0.33/0.68	0.30/0.60	0.54/1.17	0.46/0.89
STGAT w/o LM	0.61/1.05	0.32/0.61	0.33/0.66	0.28/0.58	0.52/1.11	0.41/0.80
Ours–STGAT	0.59/1.02	0.34/0.61	0.32/0.65	0.30/0.62	0.52/1.12	0.41/0.80

, where (1) STGAT w/o OF indicates no optimization features and that only the loss mask was used and (2) STGAT w/o LM indicates no loss mask and that only the optimization features were used. The results show that the use of the loss mask and the optimization features alone can both improve the prediction effectiveness of the baseline models, with the effect of using the optimization features being more obvious. Moreover, Ours–STGAT using a loss mask and optimized feature at the same time has advantages on individual datasets but does not improve on the average performance compared to STGAT w/o LM. This is because we introduce the loss mask aims to eliminate the influence of pedestrians without historical trajectories on the construction of deterministic motion patterns during network training. This improves the network’s ability to absorb and discriminate the training data, enhances its stability, and steadily improves the model prediction performance. The effect of the loss mask will be illustrated through the experiment shown in Figure 3.

Data efficiency: The size of the dataset has an impact on the effectiveness of the model. An increase in the training data tends to improve the effectiveness of the model, but a large amount of training data implies a large computational cost; therefore, in this subsection, we explore the relationship between the size of the training dataset and the learning efficiency. We conducted a series of experiments in which different percentages of training data were randomly sampled from the original training data. The training data were randomly selected and divided into 5%, 25%, and 50% samples, and the same data were input to train different models. The experiments used STGAT as a baseline model to compare two of our methods, Ours–STGAT and STGAT w/o LM, and the experimental results of the data learning efficiency with average ADE and FDE values are shown in Figure 3a and 3b, respectively. (1) It is clear that the predictive efficacy of Ours–STGAT exceeds that of the baseline model, STGAT, when using different proportions of training data; (2) STGAT w/o LM clearly exhibits the same instability as the baseline, while Ours–STGAT, with both a loss mask and an optimization feature (OF), exhibits the ideal of stable absorption of training data. It should be noted that as the amount of training data increases, both the baseline model and STGAT w/o LM exhibit unstable learning efficiency compared to Ours–STGAT. This suggests that they are not able to absorb the additional training data efficiently. We consider that this instability is precisely due to inherent interference, where the training data cannot properly and effectively guide model learning. Our proposed two-stage motion pattern de-perturbation strategy eliminates the redundant effects of introducing latent vectors and the inherent disturbances that exist when training the model. This ensures that the baseline model can effectively utilize the training data to establish accurate pedestrian motion patterns, allowing it to perform as intended and demonstrate improved results.

4.3. Visualization Presentation

To better visualize the improvement in trajectory prediction achieved by our two-stage motion pattern de-perturbation strategy compared with that achieved by the baseline models, in Figure 4 and Figure 5 we visualize the predicted trajectories of Ours–STGAT and Ours–SGAN with their corresponding baseline models in five different real-world scenarios. The overall accuracy of trajectory prediction using our method is significantly better than that of the baseline models, and the generated trajectories are more reliable. It should be noted that the visualization results for multiple scenarios indicate that the baseline models predict shorter trajectories for pedestrians who continue walking straight. Even for pedestrians with variable trajectories, the predicted trajectories tend to be closer to the actual trajectories after implementing our strategy. The reason for this is that the stochastic nature of latent vectors is not related to the pedestrians’ own motion characteristics. The introduction of latent vectors to construct trajectory multimodality in a crude and simple manner can have a redundant effect on the pedestrian’s motion pattern. This can result in irrational trajectories, as shown in the visualization figure.

5. Conclusions

In this paper, we propose a two-stage motion pattern de-perturbation strategy, introduce optimization features for eliminating the redundancy effects caused by latent vectors when constructing pedestrian motion patterns, design loss masks to decrease the interference of invalid training data, and model trajectory prediction models efficiently and accurately. Our approach is a plug-and-play module that maximizes the actual effectiveness of the baseline models and improves the trajectory prediction accuracy. Experimental results from multiple real-scene datasets demonstrate that our method significantly reduces the average trajectory prediction error across different baseline models, achieving superior prediction accuracy compared to advanced baseline models. In addition, a comparison of the experimental results with different training data confirms the universality of our method and shows that our method can indeed eliminate interference in modeling motion patterns and effectively guide a model to absorb additional training data. However, the use of latent vectors to model multimodal trajectories has limitations, as they are random numbers independent of pedestrian movement characteristics, resulting in the poor robustness of the modelled motion patterns. Future research should explore more efficient ways of modeling multimodal trajectories, such as incorporating scene information or combining multiple pedestrian motion features.