Artificial Intelligence-Enabled Traffic Monitoring System
Abstract
:1. Introduction
- Monitoring traffic congestion
- Traffic accidents, stationary or stranded vehicle detection
- Vehicle detection and count
- Managing traffic using a stand-alone Graphical User Interface (GUI)
- Scaling traffic monitoring to multiple traffic cameras.
2. Literature Review
2.1. Deep Learning Frameworks for Object Detection and Classification
2.2. Vision-Based Traffic Analysis Systems
3. Proposed Methodology
3.1. Faster R-CNN
3.2. Mask R-CNN
3.3. YOLO
3.4. CenterNet
3.5. Monitoring Traffic Queues
3.6. Detecting Stationary Vehicles
4. Data Description
5. Results
5.1. Traffic Queues Detection
A Case Study for Studying Traffic Queues
- Step 1: Extract queue regions from traffic video feeds with Mask RCNN.
- Step 2: Calculate the pixel length of each detected queue mask.
- Step 3: Accumulate length over time (minimum duration is 1 week).
- Step 4: Use adaptive thresholding (Figure 8) to bin queue lengths into different severity levels: low, medium and high.
- Step 5: Generate heat map of queuing levels and finally, compare.
5.2. Stationary Vehicle Detection
Tracking Detection by IOU and Feature Tracker
5.3. Vehicle Counts
6. Front-End Graphical User Interface
7. Conclusions
Author Contributions
Funding
Conflicts of Interest
References
- Land, E.H. An alternative technique for the computation of the designator in the retinex theory of color vision. Proc. Natl. Acad. Sci. USA 1986, 83, 3078–3080. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Rahman, Z.-u.; Jobson, D.J.; Woodell, G.A. Multi-scale retinex for color image enhancement. In Proceedings of the 3rd IEEE International Conference on Image Processing, Lausanne, Switzerland, 19 September 1996; IEEE: Piscataway, NJ, USA; pp. 1003–1006. [Google Scholar]
- He, K.; Sun, J.; Tang, X. Single image haze removal using dark channel prior. IEEE Trans. pattern Anal. Mach. Intell. 2010, 33, 2341–2353. [Google Scholar] [PubMed]
- GitHub. Video Demonstration of a GUI based AI Enabled Traffic Monitoring System. Available online: https://github.com/titanmu/aienabled (accessed on 30 September 2020).
- Willis, C.; Harborne, D.; Tomsett, R.; Alzantot, M. A Deep Convolutional Network for Traffic Congestion Classification. Available online: https://dais-ita.org/sites/default/files/nato_ist_trafficCongestion_Paper4_Issue1.pdf (accessed on 4 October 2020).
- Chakraborty, P.; Adu-Gyamfi, Y.O.; Poddar, S.; Ahsani, V.; Sharma, A.; Sarkar, S. Traffic congestion detection from camera images using deep convolution neural networks. Transp. Res. Rec. 2018, 2672, 222–231. [Google Scholar] [CrossRef] [Green Version]
- Morris, T.; Schwach, J.A.; Michalopoulos, P.G. Low-Cost Portable Video-Based Queue Detection for Work-Zone Safety; Technical Report No. 1129; Department of Civil Engineering, University of Minnesota: Minneapolis, MN, USA, 2011. [Google Scholar]
- Fouladgar, M.; Parchami, M.; Elmasri, R.; Ghaderi, A. Scalable deep traffic flow neural networks for urban traffic congestion prediction. In Proceedings of the 2017 International Joint Conference on Neural Networks (IJCNN), Anchorage, AK, USA, 14–19 May 2017; IEEE: Piscataway, NJ, USA; pp. 2251–2258. [Google Scholar]
- Ma, X.; Yu, H.; Wang, Y.; Wang, Y. Large-scale transportation network congestion evolution prediction using deep learning theory. PLoS ONE 2015, 10, e0119044. [Google Scholar] [CrossRef] [PubMed]
- Wang, J.; Gu, Q.; Wu, J.; Liu, G.; Xiong, Z. Traffic speed prediction and congestion source exploration: A deep learning method. In Proceedings of the 2016 IEEE 16th International Conference on Data Mining (ICDM), Barcelona, Spain, 12–15 December 2016; IEEE: Piscataway, NJ, USA; pp. 499–508. [Google Scholar]
- Carli, R.; Dotoli, M.; Epicoco, N.; Angelico, B.; Vinciullo, A. Automated evaluation of urban traffic congestion using bus as a probe. In Proceedings of the 2015 IEEE International Conference on Automation Science and Engineering (CASE), Gothenburg, Sweden, 24–28 August 2015; IEEE: Piscataway, NJ, USA; pp. 967–972. [Google Scholar]
- Litman, T. Developing indicators for comprehensive and sustainable transport planning. Transp. Res. Rec. 2007, 2017, 10–15. [Google Scholar] [CrossRef] [Green Version]
- He, K.; Gkioxari, G.; Dollár, P.; Girshick, R. Mask r-cnn. In Proceedings of the IEEE International conference on computer vision, Venice, Italy, 22–29 October 2017; pp. 2961–2969. [Google Scholar]
- Bochkovskiy, A.; Wang, C.-Y.; Liao, H.-Y.M. YOLOv4: Optimal Speed and Accuracy of Object Detection. Available online: https://arxiv.org/abs/2004.10934 (accessed on 4 October 2020).
- Ren, S.; He, K.; Girshick, R.; Sun, J. Faster r-cnn: Towards Real-Time Object Detection with Region Proposal Networks. Available online: https://arxiv.org/abs/1506.01497 (accessed on 4 October 2020).
- Duan, K.; Bai, S.; Xie, L.; Qi, H.; Huang, Q.; Tian, Q. Centernet: Keypoint triplets for object detection. In Proceedings of the IEEE International Conference on Computer Vision, Seoul, Korea, 27 October–2 November 2019; pp. 6569–6578. [Google Scholar]
- Rakhimkul, S.; Kim, A.; Pazylbekov, A.; Shintemirov, A. Autonomous object detection and grasping using deep learning for design of an intelligent assistive robot manipulation system. In Proceedings of the 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC), Bari, Italy, 6–9 October 2019; IEEE: Piscataway, NJ, USA; pp. 3962–3968. [Google Scholar]
- Cui, X.; Lu, C.; Wang, J. 3D semantic map construction using improved ORB-SLAM2 for mobile robot in edge computing environment. IEEE Access 2020, 8, 67179–67191. [Google Scholar] [CrossRef]
- Liu, Y.; Zhao, Z.; Chang, F.; Hu, S. An anchor-free convolutional neural network for real-time surgical tool detection in robot-assisted surgery. IEEE Access 2020, 8, 78193–78201. [Google Scholar] [CrossRef]
- Wang, D.; Zhang, N.; Sun, X.; Zhang, P.; Zhang, C.; Cao, Y.; Liu, B. AFP-Net: Realtime anchor-free polyp detection in colonoscopy. In Proceedings of the 2019 IEEE 31st International Conference on Tools with Artificial Intelligence (ICTAI), Portland, OR, USA, 4–6 November 2019; pp. 636–643. [Google Scholar]
- Chung, M.; Lee, J.; Park, S.; Lee, M.; Lee, C.E.; Lee, J.; Shin, Y.-G. Individual tooth detection and identification from dental panoramic x-ray images via point-wise localization and distance regularization. Available online: https://arxiv.org/abs/2004.05543 (accessed on 4 October 2020).
- Algabri, M.; Mathkour, H.; Bencherif, M.A.; Alsulaiman, M.; Mekhtiche, M.A. Towards deep object detection techniques for phoneme recognition. IEEE Access 2020, 8, 54663–54680. [Google Scholar] [CrossRef]
- Liu, Z.; Zheng, T.; Xu, G.; Yang, Z.; Liu, H.; Cai, D. Training-Time-Friendly Network for Real-Time Object Detection; AAAI: Menlo Park, CA, USA, 2020; pp. 11685–11692. [Google Scholar]
- Moranduzzo, T.; Melgani, F. Automatic car counting method for unmanned aerial vehicle images. IEEE Trans. Geosci. Remote Sens. 2013, 52, 1635–1647. [Google Scholar] [CrossRef]
- Kamenetsky, D.; Sherrah, J. Aerial car detection and urban understanding. In Proceedings of the 2015 International Conference on Digital Image Computing: Techniques and Applications (DICTA), Adelaide, SA, Australia, 23–25 November 2015; IEEE: Piscataway, NJ, USA; pp. 1–8. [Google Scholar]
- Arteta, C.; Lempitsky, V.; Noble, J.A.; Zisserman, A. Interactive object counting. In Proceedings of the European Conference on Computer Vision, Zurich, Switzerland, 6–12 September 2020; Springer: Berlin, Germany; pp. 504–518. [Google Scholar]
- Chiu, C.-C.; Ku, M.-Y.; Wang, C.-Y. Automatic Traffic Surveillance System for Vision-Based Vehicle Recognition and Tracking. J. Inf. Sci. Eng. 2010, 26, 611–629. [Google Scholar]
- Zhuang, P.; Shang, Y.; Hua, B. Statistical methods to estimate vehicle count using traffic cameras. Multidimens. Syst. Signal. Process. 2009, 20, 121–133. [Google Scholar] [CrossRef]
- Mundhenk, T.N.; Konjevod, G.; Sakla, W.A.; Boakye, K. A large contextual dataset for classification, detection and counting of cars with deep learning. In Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands, 11–14 October 2016; Springer: Berlin, Germany; pp. 785–800. [Google Scholar]
- Kamijo, S.; Matsushita, Y.; Ikeuchi, K.; Sakauchi, M. Traffic monitoring and accident detection at intersections. IEEE Trans. Intell. Transp. Syst. 2000, 1, 108–118. [Google Scholar] [CrossRef] [Green Version]
- Gangisetty, R. Advanced traffic management system on I-476 in Pennsylvania. In Proceedings of the Conference on Intelligent Transportation Systems, Boston, MA, USA, 12 November 1997; IEEE: Piscataway, NJ, USA; pp. 373–378. [Google Scholar]
- Rojas, J.C.; Crisman, J.D. Vehicle detection in color images. In Proceedings of the Conference on Intelligent Transportation Systems, Boston, MA, USA, 12 November 1997; IEEE: Piscataway, NJ, USA; pp. 403–408. [Google Scholar]
- Zeng, N.; Crisman, J.D. Vehicle matching using color. In Proceedings of the Conference on Intelligent Transportation Systems, Boston, MA, USA, 12 November 1997; IEEE: Piscataway, NJ, USA; pp. 206–211. [Google Scholar]
- Ai, A.H.; Yungf, N.H. A video-based system methodology for detecting red light runners. In Proceedings of the IAPR Workshop on Machine Vision Applications, Makuhari, Chiba, Japan, 17–19 November 1998. [Google Scholar]
- Thajchayapong, S.; Garcia-Trevino, E.S.; Barria, J.A. Distributed classification of traffic anomalies using microscopic traffic variables. IEEE Trans. Intell. Transp. Syst. 2012, 14, 448–458. [Google Scholar] [CrossRef] [Green Version]
- Ikeda, H.; Kaneko, Y.; Matsuo, T.; Tsuji, K. Abnormal incident detection system employing image processing technology. In Proceedings of the 199 IEEE/IEEJ/JSAI International Conference on Intelligent Transportation Systems (Cat. No. 99TH8383), Tokyo, Japan, 5–8 October 1999; pp. 748–752. [Google Scholar]
- Law, H.; Deng, J. Cornernet: Detecting objects as paired keypoints. In Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, 8–14 September 2018; pp. 734–750. [Google Scholar]
- Dutta, A.; Zisserman, A. The VIA annotation software for images, audio and video. In Proceedings of the 27th ACM International Conference on Multimedia, Nice, France, 25 November 2019; pp. 2276–2279. [Google Scholar]
- The AI City Challenge. Available online: https://www.aicitychallenge.org/ (accessed on 28 September 2020).
- Bochinski, E.; Eiselein, V.; Sikora, T. High-speed tracking-by-detection without using image information. In Proceedings of the 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Lecce, Italy, 29 August–1 September 2017; IEEE: Piscataway, NJ, USA; pp. 1–6. [Google Scholar]
- NTIMC. Benefits of Traffic Incident Management; National Traffic Incident Management Coalition: Amsterdam, The Netherlands, 2006. [Google Scholar]
- Haghani, I.; Hamedi, Y. Methodology for Quantifying the Cost Effectiveness of Freeway Service Patrol Programs—Case Study: H.E.L.P: Program; Final Report; University of Maryland: College Park, MD, USA, 2006. [Google Scholar]
- Baykal-Gürsoy, M.; Xiao, W.; Ozbay, K. Modeling traffic flow interrupted by incidents. Eur. J. Oper. Res. 2009, 195, 127–138. [Google Scholar] [CrossRef]
- Yang, H.; Ozbay, K.; Xie, K.; Ma, Y. Development of an automated approach for quantifying spatiotemporal impact of traffic incidents. In Proceedings of the Transportation Research Board 95th Annual Meeting, Washington, DC, USA, 1–14 January 2016; pp. 10–14. [Google Scholar]
- Mandal, V.; Adu-Gyamfi, Y. Object Detection and Tracking Algorithms for Vehicle Counting: A Comparative Analysis. Available online: https://arxiv.org/abs/2007.16198 (accessed on 4 October 2020).
- Mandal, V. Artificial Intelligence Enabled Automatic Traffic Monitoring System. Master’s Thesis, University of Missouri--Columbia, Columbia, MO, USA, December 2019. [Google Scholar]
- React. Available online: https://reactjs.org/ (accessed on 8 October 2020).
Model | Precision | Recall | Accuracy |
---|---|---|---|
Mask R-CNN | 92.8 | 95.6 | 90.5 |
YOLO | 95.5 | 94.8 | 93.7 |
YOLO | ||||||
Pred | Ped | Cyclist | Car | Bus | Truck | |
True | ||||||
Ped | 0.9928 | 0.0053 | 0.0008 | 0 | 0.0008 | |
Cyclist | 0.0228 | 0.9726 | 0 | 0 | 0.0045 | |
Car | 0 | 0 | 0.9947 | 0.0001 | 0.0050 | |
Bus | 0 | 0 | 0 | 0.9942 | 0.0057 | |
Truck | 0 | 0 | 0.0457 | 0.0074 | 0.9467 | |
Faster R-CNN | ||||||
Pred | Ped | Cyclist | Car | Bus | Truck | |
True | ||||||
Ped | 0.9973 | 0.0026 | 0 | 0 | 0 | |
Cyclist | 0.0401 | 0.9553 | 0.0044 | 0 | 0 | |
Car | 0.0002 | 0.0001 | 0.9943 | 0.0003 | 0.0047 | |
Bus | 0 | 0 | 0.0103 | 0.9792 | 0.0103 | |
Truck | 0 | 0 | 0.0367 | 0.0079 | 0.9553 |
YOLO | |||
---|---|---|---|
Class | Precision | Recall | F-1 Score |
Ped | 0.9216 | 0.7367 | 0.8188 |
Cyclist | 0.9424 | 0.8658 | 0.9025 |
Car | 0.9276 | 0.7990 | 0.8585 |
Bus | 0.9508 | 0.8571 | 0.9015 |
Truck | 0.9160 | 0.8400 | 0.8764 |
Total | 0.9269 | 0.7975 | 0.8573 |
Faster R-CNN | |||
Ped | 0.8754 | 0.8838 | 0.8796 |
Cyclist | 0.9380 | 0.8514 | 0.8926 |
Car | 0.8312 | 0.8788 | 0.8543 |
Bus | 0.8952 | 0.8663 | 0.8805 |
Truck | 0.8928 | 0.8972 | 0.8950 |
Total | 0.8417 | 0.8798 | 0.8604 |
Name | F1 | RMSE | S3 |
---|---|---|---|
Model M1 | 0.8333 | 154.7741 | 0.4034 |
Time of Day | Detector/Tracker Combination | Northbound Count Percentage | Southbound Count Percentage |
---|---|---|---|
Day | CenterNet and IOU | 137.04 | 144.06 |
CenterNet and Feature Tracker | 75.02 | 105.66 | |
YOLOv4 and IOU | 144.38 | 155.27 | |
YOLOv4 and Feature Tracker | 70.81 | 89.70 | |
Night | CenterNet and IOU | 144.75 | 161.38 |
CenterNet and Feature Tracker | 74.74 | 112.41 | |
YOLOv4 and IOU | 145.91 | 166.23 | |
YOLOv4 and Feature Tracker | 72.99 | 87.12 | |
Rain | CenterNet and IOU | 169.74 | 150.31 |
CenterNet and Feature Tracker | 119.14 | 99.47 | |
YOLOv4 and IOU | 145.91 | 153.76 | |
YOLOv4 and Feature Tracker | 82.06 | 74.89 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Mandal, V.; Mussah, A.R.; Jin, P.; Adu-Gyamfi, Y. Artificial Intelligence-Enabled Traffic Monitoring System. Sustainability 2020, 12, 9177. https://doi.org/10.3390/su12219177
Mandal V, Mussah AR, Jin P, Adu-Gyamfi Y. Artificial Intelligence-Enabled Traffic Monitoring System. Sustainability. 2020; 12(21):9177. https://doi.org/10.3390/su12219177
Chicago/Turabian StyleMandal, Vishal, Abdul Rashid Mussah, Peng Jin, and Yaw Adu-Gyamfi. 2020. "Artificial Intelligence-Enabled Traffic Monitoring System" Sustainability 12, no. 21: 9177. https://doi.org/10.3390/su12219177