Game-Based Flexible Merging Decision Method for Mixed Traffic of Connected Autonomous Vehicles and Manual Driving Vehicles on Urban Freeways
Abstract
:1. Introduction
1.1. Background
1.2. Related Works
1.2.1. Merging Decision-Making Methods
1.2.2. Driving Style Recognition
1.2.3. Review of Research Status
- Strategy Quantification Issue: The game’s strategy space, often based on semantic information like “yield or merge”, makes it challenging to select strategies that align with individual preferences.
- Decision Implementation Issue: Pure-strategy Nash equilibria, if used as non-allocation mechanisms, may fail to produce an equilibrium in some game rounds. Mixed-strategy Nash equilibria yield probabilistic outcomes, posing a dilemma for decision-making under these conditions.
- Incomplete Information on MDVs: Without communication tools, MDVs’ strategies, payoffs, and potential actions remain unknown, presenting a significant research challenge.
- Inadequate Modeling of MDV Behavior: Some studies presume MDVs act with complete rationality, avoiding obvious errors. Yet, in reality, drivers’ rationality is bounded, and the likelihood of mistakes, especially uncoordinated actions, can significantly impact CAVs’ merging decisions.
1.3. Motivation and Contribution
- We propose a merging decision-making method that integrates game theory for timing decisions and DRL for behavior decisions. This integration effectively addresses the challenge of making vehicle behavior decisions in the absence of game equilibrium.
- The study delves into the dynamic nature of MDVs’ driving styles, concluding that styles vary by location. A GMM-SVM-based dynamic recognition framework is introduced, adept at handling both the diversity and variability of driving styles across different locations, and effectively simulating MDV behavior.
- Analyzing the preferences of each driving style type for various payoffs, we dynamically estimate the weights for MDVs’ considerations of timeliness, safety, comfort, and economy, allowing for a more nuanced prediction of their cooperation levels.
2. Problem Description and Framework
2.1. Description of the Flexible Merger Problem
2.2. Method Architecture
3. Flexible Merging Decision Methods
3.1. Incomplete Information Static Game Models
3.1.1. Quantification of Strategies
3.1.2. Payoff Function
- (1)
- Timeliness payoff Rtime
- (2)
- Safety payoff Rsafe
- (3)
- Comfort payoff Rcomf
- (4)
- Economics payoff Reco
3.1.3. Merging Timing Decision Based on Pure-Strategy Nash Equilibrium
3.2. Motion Decision Based on DRL
- (1)
- Observation state
- (2)
- Action space
- (3)
- Reward function
- (4)
- Decision networks
4. Dynamic Driving Style Recognition Method
4.1. Data Acquisition
4.2. Driving Style Recognition Based on GMM-SVM
4.3. Calibration of Payoff Weights
5. Simulation Experiments
5.1. DRL Model Training
5.2. Micro Case Study
5.3. Macro Traffic Flow Case Study
5.4. Discussion
6. Conclusions and Future Works
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Wan, X.; Jin, P.J.; Gu, H.; Chen, X.; Ran, B. Modeling Freeway Merging in a Weaving Section as a Sequential Decision-Making Process. J. Transp. Eng. Part A Syst. 2017, 143, 05017002. [Google Scholar]
- Choudhury, C.F.; Ramanujam, V.; Ben-Akiva, M.E. Modeling Acceleration Decisions for Freeway Merges. Transp. Res. Rec. 2009, 2124, 45–57. [Google Scholar]
- Lu, X.-Y.; Hedrick, K.J. Longitudinal Control Algorithm for Automated Vehicle Merging. In Proceedings of the 39th IEEE Conference on Decision and Control, Sydney, Australia, 12–15 December 2000; Volume 5, pp. 450–455. [Google Scholar]
- Lu, X.-Y.; Tan, H.-S.; Shladover, S.E.; Hedricket, J.K. Implementation of Longitudinal Control Algorithm for Vehicle Merging. In Proceedings of the 5th International Symposium on Advanced Vehicle Control, Ann Arbor, MI, USA, 22–24 August 2000; pp. 25–32. [Google Scholar]
- Yang, C.; Kurami, K. Longitudinal guidance and control for the entry of vehicles onto automated highways. In Proceedings of the 32nd IEEE Conference on Decision and Control, San Antonio, TX, USA, 15–17 December 1993; Volume 2, pp. 1891–1896. [Google Scholar]
- He, X.; Lv, C. Toward personalized decision making for autonomous vehicles: A constrained multi-objective reinforcement learning technique. Transp. Res. Part C Emerg. Technol. 2023, 156, 104352. [Google Scholar]
- el abidine Kherroubi, Z.; Aknine, S.; Bacha, R. Novel Decision-Making Strategy for Connected and Autonomous Vehicles in Highway On-Ramp Merging. IEEE Trans. Intell. Transp. Syst. 2022, 23, 12490–12502. [Google Scholar]
- Lin, Y.; McPhee, J.; Azad, N.L. Anti-Jerk On-Ramp Merging Using Deep Reinforcement Learning. In Proceedings of the 2020 IEEE Intelligent Vehicles Symposium (IV), Las Vegas, NV, USA, 19 October–13 November 2020; pp. 7–14. [Google Scholar]
- Wang, M.; Hoogendoorn, S.P.; Daamen, W.; van Arem, B.; Happee, R. Game theoretic approach for predictive lane-changing and car-following control. Transp. Res. Part C Emerg. Technol. 2015, 58, 73–92. [Google Scholar]
- Chen, W.; Ren, G.; Cao, Q.; Song, J.; Liu, Y.; Dong, C. A Game-Theory-Based Approach to Modeling Lane-Changing Interactions on Highway On-Ramps: Considering the Bounded Rationality of Drivers. Mathematics 2023, 11, 402. [Google Scholar] [CrossRef]
- Liu, J.; Zhao, W.; Wang, C.; Zhou, Y.; Zhang, Z.; Qin, Y. Interactive on-ramp merging decision-making with motion prediction considering merging motivation. Proc. Inst. Mech. Eng. Part D J. Automob. Eng. 2023. [Google Scholar] [CrossRef]
- Fukuyama, S. Dynamic game-based approach for optimizing merging vehicle trajectories using time-expanded decision diagram. Transp. Res. Part C Emerg. Technol. 2020, 120, 102766. [Google Scholar]
- Li, W.; Qiu, F.; Li, L.; Zhang, Y.; Wang, K. Simulation of Vehicle Interaction Behavior in Merging Scenarios: A Deep Maximum Entropy-Inverse Reinforcement Learning Method Combined with Game Theory. IEEE Trans. Intell. Veh. 2024, 9, 1079–1093. [Google Scholar]
- Liu, K.; Li, N.; Tseng, H.E.; Kolmanovsky, I.; Girard, A.; Filev, D. Cooperation-Aware Decision Making for Autonomous Vehicles in Merge Scenarios. In Proceedings of the 60th IEEE Conference on Decision and Control (CDC), Austin, TX, USA, 13–17 December 2021; pp. 5006–5012. [Google Scholar]
- Ding, H.; Di, Y.; Zheng, X.; Bai, H.; Zhang, W. Automated cooperative control of multilane freeway merging areas in connected and autonomous vehicle environments. Transp. B Transp. Dyn. 2021, 9, 437–455. [Google Scholar]
- Xie, Y.; Zhang, H.; Gartner, N.H.; Arsava, T. Collaborative merging strategy for freeway ramp operations in a connected and autonomous vehicles environment. J. Intell. Transp. Syst. 2017, 21, 136–147. [Google Scholar]
- Pei, H.; Feng, S.; Zhang, Y.; Yao, D. A Cooperative Driving Strategy for Merging at On-Ramps Based on Dynamic Programming. IEEE Trans. Veh. Technol. 2019, 68, 11646–11656. [Google Scholar]
- Shi, J.; Li, K.; Chen, C.; Kong, W.; Luo, Y. Cooperative Merging Strategy in Mixed Traffic Based on Optimal Final-State Phase Diagram with Flexible Highway Merging Points. IEEE Trans. Intell. Transp. Syst. 2023, 24, 11185–11197. [Google Scholar]
- Ding, J.; Li, L.; Peng, H.; Zhang, Y. A Rule-Based Cooperative Merging Strategy for Connected and Automated Vehicles. IEEE Trans. Intell. Transp. Syst. 2020, 21, 3436–3446. [Google Scholar]
- Xu, H.; Feng, S.; Zhang, Y.; Li, L. A Grouping-Based Cooperative Driving Strategy for CAVs Merging Problems. IEEE Trans. Veh. Technol. 2019, 68, 6125–6136. [Google Scholar]
- Yang, W.; Dong, C.; Wang, H. A cooperative merging speed control strategy of CAVs based on virtual platoon in on-ramp merging system. Transp. B Transp. Dyn. 2023, 11, 1432–1454. [Google Scholar]
- Hu, X.; Sun, J. Trajectory optimization of connected and autonomous vehicles at a multilane freeway merging area. Transp. Res. Part C Emerg. Technol. 2019, 101, 111–125. [Google Scholar]
- Shi, Y.; Yu, H.; Guo, Y.; Yuan, Z. A Collaborative Merging Strategy with Lane Changing in Multilane Freeway On-Ramp Area with V2X Network. Futur. Internet 2021, 13, 123. [Google Scholar]
- Han, L.; Zhang, L.; Guo, W. Multilane freeway merging control via trajectory optimization in a mixed traffic environment. IET Intell. Transp. Syst. 2023, 17, 1891–1907. [Google Scholar]
- Elander, J.; West, R.; French, D. Behavioral correlates of individual differences in road-traffic crash risk: An examination of methods and findings. Psychol. Bull. 1993, 113, 279. [Google Scholar]
- Woo, H.; Ji, Y.; Tamura, Y.; Kuroda, Y.; Sugano, T.; Yamamoto, Y.; Yamashita, A.; Asama, H. Dynamic state estimation of driving style based on driving risk feature. Int. J. Automot. Eng. 2018, 9, 31–38. [Google Scholar]
- Sysoev, M.; Kos, A.; Guna, J.; Pogačnik, M. Estimation of the driving style based on the users’ activity and environment influence. Sensors 2017, 17, 2404. [Google Scholar] [CrossRef]
- Han, W.; Wang, W.; Li, X.; Xi, J. Statistical-based approach for driving style recognition using Bayesian probability with kernel density estimation. IET Intell. Transp. Syst. 2019, 13, 22–30. [Google Scholar]
- Reason, J.; Manstead, A.; Stradling, S.; Baxter, J.; Campbell, K. Errors and violations on the roads: A real distinction? Ergonomics 1990, 33, 1315–1332. [Google Scholar] [PubMed]
- Taubman-Ben-Ari, O.; Mikulincer, M.; Gillath, O. The multidimensional driving style inventory—Scale construct and validation. Accid. Anal. Prev. 2004, 36, 323–332. [Google Scholar] [PubMed]
- Ishibashi, M.; Okuwa, M.; Doi, S.I.; Akamatsu, M. Indices for characterizing driving style and their relevance to car following behavior. In Proceedings of the SICE Annual Conference, Takamatsu, Japan, 17–20 September 2007; IEEE: Piscataway, NJ, USA, 2007; pp. 1132–1137. [Google Scholar]
- Chen, S.-W.; Fang, C.-Y.; Tien, C.-T. Driving behaviour modelling system based on graph construction. Transp. Res. Part C Emerg. Technol. 2013, 26, 314–330. [Google Scholar]
- Wang, W.; Xi, J.; Chen, H. Modeling and recognizing driver behavior based on driving data: A survey. Math. Probl. Eng. 2014, 2014, 245641. [Google Scholar]
- Chu, D.; Deng, Z.; He, Y.; Wu, C.; Sun, C.; Lu, Z. Curve speed model for driver assistance based on driving style classification. IET Intell. Transp. Syst. 2017, 11, 501–510. [Google Scholar]
- Bellem, H.; Thiel, B.; Schrauf, M.; Krems, J.F. Comfort in automated driving: An analysis of preferences for different automated driving styles and their dependence on personality traits. Transp. Res. Part F Traffic Psychol. Behav. 2018, 55, 90–100. [Google Scholar]
- Van Huysduynen, H.H.; Terken, J.; Eggen, B. The relation between self-reported driving style and driving behaviour. A simulator study. Transp. Res. Part F Traffic Psychol. Behav. 2018, 56, 245–255. [Google Scholar]
- Murphey, Y.L.; Milton, R.; Kiliaris, L. Driver’s style classification using jerk analysis. In Proceedings of the 2009 IEEE Workshop on Computational Intelligence in Vehicles and Vehicular Systems, Nashville, TN, USA, 30 March–2 April 2009; IEEE: Piscataway, NJ, USA, 2009; pp. 23–28. [Google Scholar]
- Wang, W.; Xi, J. A rapid pattern-recognition method for driving styles using clustering-based support vector machines. In Proceedings of the 2016 American Control Conference (ACC), Boston, MA, USA, 6–8 July 2016; IEEE: Piscataway, NJ, USA, 2016; pp. 5270–5275. [Google Scholar]
- Mantouka, E.G.; Barmpounakis, E.N.; Vlahogianni, E.I. Identifying driving safety profiles from smartphone data using unsupervised learning. Saf. Sci. 2019, 119, 84–90. [Google Scholar]
- Zhang, H.; Fu, R. An Ensemble Learning—Online Semi-Supervised Approach for Vehicle Behavior Recognition. IEEE Trans. Intell. Transp. Syst. 2022, 23, 10610–10626. [Google Scholar]
- Hu, Y.; Jin, X.; Wang, Y.; Guo, J.; Zhang, L.; Hu, J.; Lu, Q. Eco-Driving of Connected and Automated Vehicles in Mixed and Power-Heterogeneous Traffic Flow. China J. Highw. Transp. 2022, 35, 15–27. [Google Scholar]
- Van Hasselt, H.; Guez, A.; Silver, D. Deep Reinforcement Learning with Double Q-Learning. In Proceedings of the 30th Association-for-the-Advancement-of-Artificial-Intelligence (AAAI), Conference on Artificial Intelligence, Phoenix, AZ, USA, 12–17 February 2016; pp. 2094–2100. [Google Scholar]
- Schaul, T.; Quan, J.; Antonoglou, I.; Silver, D. Prioritized Experience Replay. In Proceedings of the 4th International Conference on Learning Representations (ICLR), San Juan, PR, USA, 2–4 May 2016. [Google Scholar]
- Suzuki, K.; Nakamura, H. Development and performance evaluation of a video image processing system for traffic flow analysis–traffic analyzer. In Proceedings of the Civil Society Proceedings, Tokyo, Japan, 25 October 2006; pp. 276–287. (In Japanese). [Google Scholar]
- Dou, H.; Liu, Y.; Chen, S.; Zhao, H.; Bilal, H. A hybrid CEEMD-GMM scheme for enhancing the detection of traffic flow on highways. Soft Comput. 2023, 27, 16373–16388. [Google Scholar]
- Chauhan, V.K.; Dahiya, K.; Sharma, A. Problem formulations and solvers in linear SVM: A review. Artif. Intell. Rev. 2019, 52, 803–855. [Google Scholar]
Allow | Not Allow | |
---|---|---|
Merge | CAVs go first | Risk of collision |
Not Merge | Deadlock | MDVs go first |
Allow | Not Allow | |
---|---|---|
Merge | ||
Not Merge |
Location | Style | Accurate | ||||
---|---|---|---|---|---|---|
MZ1 | Aggressive | 35.09 | 0.38 | 0.30 | 5.18 | 93.33% |
Normal | 29.34 | 0.37 | 0.64 | 4.40 | 90.12% | |
Conservative | 30.66 | 0.55 | 0.51 | 3.46 | 91.27% | |
MZ2 | Aggressive | 36.81 | −0.80 | 0.38 | 3.19 | 99.76% |
Normal | 32.72 | −0.20 | 0.33 | 3.24 | 94.38% | |
Conservative | 27.76 | 0.08 | 0.64 | 4.93 | 95.96% | |
MZ3 | Aggressive | 35.73 | −0.46 | 0.34 | 3.07 | 98.63% |
Normal | 29.56 | −0.05 | 0.46 | 2.83 | 95.34% | |
Conservative | 27.23 | 0.31 | 0.69 | 4.75 | 93.33% |
Location | Style | ||||
---|---|---|---|---|---|
MZ1 | Aggressive | 0.21 | 0.26 | 0.18 | 0.35 |
Conservative | 0.26 | 0.25 | 0.29 | 0.20 | |
Normal | 0.21 | 0.27 | 0.25 | 0.27 | |
MZ2 | Aggressive | 0.25 | 0.31 | 0.22 | 0.22 |
Conservative | 0.29 | 0.26 | 0.30 | 0.16 | |
Normal | 0.27 | 0.29 | 0.25 | 0.19 | |
MZ3 | Aggressive | 0.24 | 0.22 | 0.19 | 0.34 |
Conservative | 0.26 | 0.20 | 0.31 | 0.24 | |
Normal | 0.23 | 0.20 | 0.23 | 0.35 |
Vehicle ID | L | Speed |
---|---|---|
Af | 70 m | 11 m/s |
A | 90 m | 10 m/s |
Ab | 125 m | 9 m/s |
Bf | 80 m | 11 m/s |
B | 95 m | 10 m/s |
Bb | 125 m | 9 m/s |
Success Rate | Average Speed (km/h) | Average Minimum Distance (m) | Speed Perturbation | |
---|---|---|---|---|
Non-control | 37.6% | 50.795 | 11.83 | gentle |
Game-based | 52.4% | 51.456 | 10.64 | severe |
DRL+Game | 85.7% | 51.935 | 11.23 | gentle |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Du, Z.; Xie, H.; Zhai, P.; Yuan, S.; Li, Y.; Wang, J.; Wang, J.; Liu, K. Game-Based Flexible Merging Decision Method for Mixed Traffic of Connected Autonomous Vehicles and Manual Driving Vehicles on Urban Freeways. Appl. Sci. 2024, 14, 7375. https://doi.org/10.3390/app14167375
Du Z, Xie H, Zhai P, Yuan S, Li Y, Wang J, Wang J, Liu K. Game-Based Flexible Merging Decision Method for Mixed Traffic of Connected Autonomous Vehicles and Manual Driving Vehicles on Urban Freeways. Applied Sciences. 2024; 14(16):7375. https://doi.org/10.3390/app14167375
Chicago/Turabian StyleDu, Zhibin, Hui Xie, Pengyu Zhai, Shoutong Yuan, Yupeng Li, Jiao Wang, Jiangbo Wang, and Kai Liu. 2024. "Game-Based Flexible Merging Decision Method for Mixed Traffic of Connected Autonomous Vehicles and Manual Driving Vehicles on Urban Freeways" Applied Sciences 14, no. 16: 7375. https://doi.org/10.3390/app14167375