Dynamic Graph Representation Learning for Passenger Behavior Prediction
Abstract
:1. Introduction
- A continuous-time dynamic graph model is proposed for the passenger behavior prediction problem. Unlike previous sequential models, DyGPP treats fine-grained timestamped events as a dynamic graph and learns temporal patterns for both passengers and stations from their historical sampling sequences.
- A temporal pattern encoding module is designed to simultaneously capture the individual temporal patterns for passengers and stations and correlate temporal behaviors between passengers and stations.
- Experiments on real-world datasets demonstrated the effectiveness of our method. Compared to the baseline models, our approach achieved higher average precision and AUC values, demonstrating its superiority.
2. Related Works
2.1. Passenger Behavior Prediction
2.2. Dynamic Graph Learning
2.3. Time-Based Batch
3. Preliminaries
3.1. Definitions
3.2. Problem Formalization
4. Methodology
4.1. Temporal Sequence Construction
4.1.1. Extracting Historical Sequences
4.1.2. Feature Encoder
4.2. Dynamic Representation Learning
4.3. Future Behavior Prediction
4.4. Parameter Optimization Algorithm
Algorithm 1: Training process of DyGPP |
5. Experiments
5.1. Dataset
5.2. Baselines
5.3. Evaluation Metrics
5.4. Implementation Details
5.5. Performance Comparison
5.6. Ablation Study
5.7. Temporal Encoder Analysis
5.8. Parameter Sensitivity
6. Discussion
7. Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Jia, R.; Li, Z.; Xia, Y.; Zhu, J.; Ma, N.; Chai, H.; Liu, Z. Urban road traffic condition forecasting based on sparse ride-hailing service data. IET Intell. Transp. Syst. 2020, 14, 668–674. [Google Scholar] [CrossRef]
- Tsai, T.H.; Lee, C.K.; Wei, C.H. Neural network based temporal feature models for short-term railway passenger demand forecasting. Expert Syst. Appl. 2009, 36, 3728–3736. [Google Scholar] [CrossRef]
- Liu, L.; Chen, R.C.; Zhu, S. Impacts of weather on short-term metro passenger flow forecasting using a deep LSTM neural network. Appl. Sci. 2020, 10, 2962. [Google Scholar] [CrossRef]
- Lijuan, Y.; Zhang, S.; Guocai, L. Neural Network-Based Passenger Flow Prediction: Take a Campus for Example. In Proceedings of the 2020 13th International Symposium on Computational Intelligence and Design (ISCID), Hangzhou, China, 12–13 December 2020; pp. 384–387. [Google Scholar]
- Tang, L.; Zhao, Y.; Cabrera, J.; Ma, J.; Tsui, K.L. Forecasting Short-Term Passenger Flow: An Empirical Study on Shenzhen Metro. IEEE Trans. Intell. Transp. Syst. 2019, 20, 3613–3622. [Google Scholar] [CrossRef]
- Gu, J.; Jiang, Z.; Fan, W.; Chen, J. Short-term trajectory prediction for individual metro passengers integrating diverse mobility patterns with adaptive location-awareness. Inform. Sci. 2022, 599, 25–43. [Google Scholar] [CrossRef]
- McFadden, D. The measurement of urban travel demand. J. Public Econ. 1974, 3, 303–328. [Google Scholar] [CrossRef]
- Wei, Y.; Chen, M.C. Forecasting the short-term metro passenger flow with empirical mode decomposition and neural networks. Transp. Res. Part Emerg. Technol. 2012, 21, 148–162. [Google Scholar] [CrossRef]
- Menon, A.K.; Lee, Y. Predicting short-term public transport demand via inhomogeneous Poisson processes. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, Singapore, 6–10 November 2017; pp. 2207–2210. [Google Scholar]
- Zhai, H.; Tian, R.; Cui, L.; Xu, X.; Zhang, W. A Novel Hierarchical Hybrid Model for Short-Term Bus Passenger Flow Forecasting. J. Adv. Transp. 2020, 2020, 7917353. [Google Scholar] [CrossRef]
- Hao, S.; Lee, D.H.; Zhao, D. Sequence to sequence learning with attention mechanism for short-term passenger flow prediction in large-scale metro system. Transp. Res. Part Emerg. Technol. 2019, 107, 287–300. [Google Scholar] [CrossRef]
- Barros, C.D.; Mendonça, M.R.; Vieira, A.B.; Ziviani, A. A survey on embedding dynamic graphs. ACM Comput. Surv. (CSUR) 2021, 55, 1–37. [Google Scholar] [CrossRef]
- Kazemi, S.M.; Goel, R.; Jain, K.; Kobyzev, I.; Sethi, A.; Forsyth, P.; Poupart, P. Representation learning for dynamic graphs: A survey. J. Mach. Learn. Res. 2020, 21, 1–73. [Google Scholar]
- You, J.; Du, T.; Leskovec, J. ROLAND: Graph learning framework for dynamic graphs. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Washington, DC, USA, 14–18 August 2022; pp. 2358–2366. [Google Scholar]
- Zhang, K.; Cao, Q.; Fang, G.; Xu, B.; Zou, H.; Shen, H.; Cheng, X. Dyted: Disentangled representation learning for discrete-time dynamic graph. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Long Beach, CA, USA, 6–10 August 2023; pp. 3309–3320. [Google Scholar]
- Nguyen, G.H.; Lee, J.B.; Rossi, R.A.; Ahmed, N.K.; Koh, E.; Kim, S. Dynamic network embeddings: From random walks to temporal random walks. In Proceedings of the 2018 IEEE International Conference on Big Data (Big Data), Seattle, WA, USA, 10–13 December 2018; pp. 1085–1092. [Google Scholar]
- Yu, W.; Cheng, W.; Aggarwal, C.C.; Zhang, K.; Chen, H.; Wang, W. Netwalk: A flexible deep embedding approach for anomaly detection in dynamic networks. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, AK, USA, 4–8 August 2018; pp. 2672–2681. [Google Scholar]
- Rossi, E.; Chamberlain, B.; Frasca, F.; Eynard, D.; Monti, F.; Bronstein, M. Temporal graph networks for deep learning on dynamic graphs. arXiv 2020, arXiv:2006.10637. [Google Scholar]
- Kumar, S.; Zhang, X.; Leskovec, J. Predicting dynamic embedding trajectory in temporal interaction networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, AK, USA, 4–8 August 2019; pp. 1269–1278. [Google Scholar]
- Tolstikhin, I.O.; Houlsby, N.; Kolesnikov, A.; Beyer, L.; Zhai, X.; Unterthiner, T.; Yung, J.; Steiner, A.; Keysers, D.; Uszkoreit, J.; et al. Mlp-mixer: An all-mlp architecture for vision. Adv. Neural Inf. Process. Syst. 2021, 34, 24261–24272. [Google Scholar]
- Wang, L.; Chang, X.; Li, S.; Chu, Y.; Li, H.; Zhang, W.; He, X.; Song, L.; Zhou, J.; Yang, H. Tcl: Transformer-based dynamic graph modelling via contrastive learning. arXiv 2021, arXiv:2105.07944. [Google Scholar]
- Yu, L.; Sun, L.; Du, B.; Lv, W. Towards better dynamic graph learning: New architecture and unified library. Adv. Neural Inf. Process. Syst. 2023, 36, 67686–67700. [Google Scholar]
- Xu, D.; Ruan, C.; Korpeoglu, E.; Kumar, S.; Achan, K. Inductive representation learning on temporal graphs. arXiv 2020, arXiv:2002.07962. [Google Scholar]
- Poursafaei, F.; Huang, S.; Pelrine, K.; Rabbany, R. Towards better evaluation for dynamic link prediction. Adv. Neural Inf. Process. Syst. 2022, 35, 32928–32941. [Google Scholar]
- Ruder, S. An overview of gradient descent optimization algorithms. arXiv 2016, arXiv:1609.04747. [Google Scholar]
- Darken, C.; Chang, J.; Moody, J. Learning rate schedules for faster stochastic gradient search. In Proceedings of the Neural Networks for Signal Processing, Helsingoer, Denmark, 31 August–2 September 1992; Volume 2, pp. 3–12. [Google Scholar]
- Bengio, Y.; Louradour, J.; Collobert, R.; Weston, J. Curriculum learning. In Proceedings of the 26th Annual International Conference on Machine Learning, Montreal, QC, Canada, 14–18 June 2009; pp. 41–48. [Google Scholar]
- Kingma, D.P.; Ba, J. Adam: A method for stochastic optimization. arXiv 2014, arXiv:1412.6980. [Google Scholar]
- Lampert, M.; Blöcker, C.; Scholtes, I. From Link Prediction to Forecasting: Information Loss in Batch-based Temporal Graph Learning. arXiv 2024, arXiv:2406.04897. [Google Scholar]
- Cheng, Z.; Trépanier, M.; Sun, L. Incorporating travel behavior regularity into passenger flow forecasting. Transp. Res. Part Emerg. Technol. 2021, 128, 103200. [Google Scholar] [CrossRef]
- Cantelmo, G.; Qurashi, M.; Prakash, A.A.; Antoniou, C.; Viti, F. Incorporating trip chaining within online demand estimation. Transp. Res. Procedia 2019, 38, 462–481. [Google Scholar] [CrossRef]
- Cong, W.; Zhang, S.; Kang, J.; Yuan, B.; Wu, H.; Zhou, X.; Tong, H.; Mahdavi, M. Do we really need complicated model architectures for temporal networks? arXiv 2023, arXiv:2302.11636. [Google Scholar]
- Karypis, G. Evaluation of Item-Based Top-N Recommendation Algorithms. In Proceedings of the 2001 ACM CIKM International Conference on Information and Knowledge Management, Atlanta, GO, USA, 5–10 November 2001; pp. 247–254. [Google Scholar]
- Hochreiter, S.; Schmidhuber, J. Long short-term memory. Neural Comput. 1997, 9, 1735–1780. [Google Scholar] [CrossRef] [PubMed]
- Cho, K.; Van Merriënboer, B.; Gulcehre, C.; Bahdanau, D.; Bougares, F.; Schwenk, H.; Bengio, Y. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv 2014, arXiv:1406.1078. [Google Scholar]
- Meyes, R.; Lu, M.; de Puiseau, C.W.; Meisen, T. Ablation studies in artificial neural networks. arXiv 2019, arXiv:1901.08644. [Google Scholar]
Ref | Time Series Prediction | Passenger Flow Prediction | Solution |
---|---|---|---|
[1] | ✓ | ✗ | Neural network models |
[2] | ✗ | ✓ | Neural network models with temporal features |
[3] | ✗ | ✓ | LSTM models |
[4] | ✗ | ✓ | CNN models |
[5] | ✓ | ✓ | ARIMA model with weather conditions |
[6] | ✗ | ✗ | StTP model with adaptive location awareness |
[8] | ✗ | ✓ | NN models with six time steps before prediction |
[9] | ✗ | ✓ | Non-homogeneous Poisson process |
[10] | ✓ | ✓ | Hierarchical hybrid model based on time series models |
[11] | ✓ | ✗ | End-to-end framework for multi-step prediction |
Notation | Meaning |
---|---|
The passenger | |
The station | |
Collections of passengers | |
Collections of stations | |
The label, which is 0 when the passenger gets on the train and 1 when they get off the train | |
The timestamp | |
The interaction | |
The dynamic graph at time t | |
Feature of the passenger | |
Feature of the station | |
The probability of interaction between passenger u and stations s at time t | |
FC | Fully connected layer |
ReLU | Linear rectification function, a type of activation function in neural networks |
Sigmoid | A common S-shaped function in biology, it also acts as the activation function for neural networks, mapping variables between 0 and 1 |
Dataset | Passenger | Station | Interaction | Interaction Per Passenger |
---|---|---|---|---|
BJSubway-40K | 459 | 392 | 44,852 | 97 |
BJSubway-1M | 7491 | 392 | 958,076 | 127 |
Parameter | Value |
---|---|
Time gap for batch | 1000 s |
Number of neighbors | 20 |
Neighbor sampling strategy | Most recent neighbor |
Negative sample strategy | Random |
Time scaling factor | 0.000001 |
Number of heads | 2 |
Number of encoder model layers | 1 |
Channel embedding dimension | 50 |
Learning rate | 0.0001 |
Max input sequence | 32 |
Datasets | BJSubway-40K | BJSubway-1M | ||
---|---|---|---|---|
Metrics | AP | AUC | AP | AUC |
TOP | 0.7880 ± 0.0000 | 0.7831 ± 0.0000 | 0.8501 ± 0.0000 | 0.8204 ± 0.0000 |
Personal TOP | 0.7545 ± 0.0000 | 0.7701 ± 0.0000 | 0.7901 ± 0.0000 | 0.8067 ± 0.0000 |
LSTM | 0.9026 ± 0.0039 | 0.8752 ± 0.0041 | 0.9395 ± 0.0015 | 0.9336 ± 0.0015 |
GRU | 0.9119 ± 0.0059 | 0.8860 ± 0.0062 | 0.9418 ± 0.0016 | 0.9357 ± 0.0018 |
TGN | 0.9390 ± 0.0032 | 0.9215 ± 0.0040 | 0.9430 ± 0.0017 | 0.9365 ± 0.0023 |
GraphMixer | 0.8565 ± 0.0078 | 0.8281 ± 0.0079 | 0.7591 ± 0.0015 | 0.7716 ± 0.0018 |
TGAT | 0.7094 ± 0.0095 | 0.6410 ± 0.0116 | 0.6563 ± 0.0107 | 0.6420 ± 0.0180 |
DyGPP | 0.9699 ± 0.0008 | 0.9591 ± 0.0015 | 0.9697 ± 0.0002 | 0.9647 ± 0.0002 |
Methods | AP | AUC |
---|---|---|
w/o eE | 0.9681 ± 0.0005 | 0.9562 ± 0.0005 |
w/o tE | 0.9685 ± 0.0010 | 0.9570 ± 0.0016 |
w/o coE | 0.8877 ± 0.0072 | 0.8601 ± 0.0085 |
w/o cosE | 0.9672 ± 0.0014 | 0.9543 ± 0.0022 |
w/o cocE | 0.8824 ± 0.0114 | 0.8533 ± 0.0135 |
DyGPP | 0.9699 ± 0.0008 | 0.9591 ± 0.0015 |
Methods | AP | AUC | Time Cost (s) |
---|---|---|---|
Transformer | 0.9692 ± 0.0006 | 0.9578 ± 0.0010 | 4707.03 |
LSTM | 0.9693 ± 0.0009 | 0.9580 ± 0.0018 | 4008.01 |
DyGPP-mamba | 0.9690 ± 0.0010 | 0.9581 ± 0.0015 | 5641.16 |
DyGPP | 0.9699 ± 0.0008 | 0.9591 ± 0.0015 | 3604.43 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Xie, M.; Zou, T.; Ye, J.; Du, B.; Huang, R. Dynamic Graph Representation Learning for Passenger Behavior Prediction. Future Internet 2024, 16, 295. https://doi.org/10.3390/fi16080295
Xie M, Zou T, Ye J, Du B, Huang R. Dynamic Graph Representation Learning for Passenger Behavior Prediction. Future Internet. 2024; 16(8):295. https://doi.org/10.3390/fi16080295
Chicago/Turabian StyleXie, Mingxuan, Tao Zou, Junchen Ye, Bowen Du, and Runhe Huang. 2024. "Dynamic Graph Representation Learning for Passenger Behavior Prediction" Future Internet 16, no. 8: 295. https://doi.org/10.3390/fi16080295