Diffusion Models-Based Purification for Common Corruptions on Robust 3D Object Detection
Abstract
:1. Introduction
- We demonstrate that existing standard adversarial training has significant limitations for 3D object detection models because of the large-scale space, sparsity, variety of corruptions, and gradient confusion.
- We propose LiDARPure, which leverages the powerful generative ability of diffusion models to purify common corruptions on LiDAR scenes and works independently of the 3D object detector.
- We propose a denoising recovery loss term, which utilizes the latent features and geometry information to supervise the reverse diffusion process.
- The experiments demonstrate that the proposed method achieves superior robustness to corruptions and outperforms existing 3D point cloud purification methods.
2. Related Work
2.1. LiDAR-Based 3D Object Detection
2.2. Adversarial Attack against 3D Object Detection
2.3. Defend Strategies for 3D Computer Vision Methods
2.4. Diffusion Models for 3D Data
3. Limitations of Adversarial Training on 3D Object Detection
4. LiDARPure: Purification for LiDAR Scenes
4.1. Preliminaries
4.2. Problem Formulation
4.3. Design of LiDARPure
4.4. Loss Function
5. Experiments
5.1. Experimental Setup
5.2. Comparison with State-of-the-Art Methods
5.3. Visualization
5.4. Ablation Study
6. Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
Abbreviations
LiDAR | Light Detection and Ranging |
GANs | Generative adversarial networks |
VAE | Variational autoencode |
NMS | Non-Maximum Suppression |
AP | Average Precision |
DDPM | Denoising Diffusion Probabilistic Models |
References
- Fernandes, D.; Silva, A.; Névoa, R.; Simoes, C.; Gonzalez, D.; Guevara, M.; Novais, P.; Monteiro, J.; Melo-Pinto, P. Point-cloud based 3D object detection and classification methods for self-driving applications: A survey and taxonomy. Inf. Fusion 2021, 68, 161–191. [Google Scholar] [CrossRef]
- Szegedy, C.; Zaremba, W.; Sutskever, I.; Bruna, J.; Erhan, D.; Goodfellow, I.J.; Fergus, R. Intriguing properties of neural networks. In Proceedings of the International Conference on Learning Representations, Banff, AB, Canada, 14–16 April 2014. [Google Scholar]
- Moosavi-Dezfooli, S.M.; Fawzi, A.; Frossard, P. DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; IEEE Computer Society: Washington, DC, USA, 2016; pp. 2574–2582. [Google Scholar] [CrossRef]
- Tu, J.; Ren, M.; Manivasagam, S.; Liang, M.; Yang, B.; Du, R.; Cheng, F.; Urtasun, R. Physically Realizable Adversarial Examples for LiDAR Object Detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 13–19 June 2020; Computer Vision Foundation/IEEE: Piscataway, NJ, USA, 2020; pp. 13713–13722. [Google Scholar] [CrossRef]
- Wang, X.; Cai, M.; Sohel, F.; Sang, N.; Chang, Z. Adversarial point cloud perturbations against 3D object detection in autonomous driving systems. Neurocomputing 2021, 466, 27–36. [Google Scholar] [CrossRef]
- Dong, Y.; Kang, C.; Zhang, J.; Zhu, Z.; Wang, Y.; Yang, X.; Su, H.; Wei, X.; Zhu, J. Benchmarking Robustness of 3D Object Detection to Common Corruptions in Autonomous Driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada, 17–24 June 2023; IEEE: Piscataway, NJ, USA, 2023; pp. 1022–1032. [Google Scholar] [CrossRef]
- Madry, A.; Makelov, A.; Schmidt, L.; Tsipras, D.; Vladu, A. Towards Deep Learning Models Resistant to Adversarial Attacks. In Proceedings of the International Conference on Learning Representations, Vancouver, BC, Canada, 30 April–3 May 2018; OpenReview.net: Amherst, MA, USA, 2018. [Google Scholar]
- Gowal, S.; Rebuffi, S.A.; Wiles, O.; Stimberg, F.; Calian, D.A.; Mann, T.A. Improving Robustness using Generated Data. In Proceedings of the Annual Conference on Neural Information Processing Systems, Virtual, 6–14 December 2021; pp. 4218–4233. [Google Scholar]
- Shi, C.; Holtz, C.; Mishne, G. Online Adversarial Purification based on Self-supervised Learning. In Proceedings of the International Conference on Learning Representations, Virtual Event, 3–7 May 2021; OpenReview.net: Amherst, MA, USA, 2021. [Google Scholar]
- Yoon, J.; Hwang, S.J.; Lee, J. Adversarial Purification with Score-based Generative Models. In Proceedings of the International Conference on Machine Learning—PMLR, Virtual Event, 18–24 July 2021; Volume 139, pp. 12062–12072. [Google Scholar]
- Croce, F.; Hein, M. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In Proceedings of the International Conference on Machine Learning—PMLR, Virtual Event, 13–18 July 2020; Volume 119, pp. 2206–2216. [Google Scholar]
- Goodfellow, I.J.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.C.; Bengio, Y. Generative Adversarial Nets. In Proceedings of the Annual Conference on Neural Information Processing Systems, Montreal, QC, Canada, 8–13 December 2014; pp. 2672–2680. [Google Scholar]
- Kingma, D.P.; Welling, M. Auto-Encoding Variational Bayes. In Proceedings of the International Conference on Learning Representations, Banff, AB, Canada, 14–16 April 2014. [Google Scholar]
- Ho, J.; Jain, A.; Abbeel, P. Denoising Diffusion Probabilistic Models. In Proceedings of the Annual Conference on Neural Information Processing Systems, Virtual, 6–12 December 2020. [Google Scholar]
- Song, Y.; Sohl-Dickstein, J.; Kingma, D.P.; Kumar, A.; Ermon, S.; Poole, B. Score-Based Generative Modeling through Stochastic Differential Equations. In Proceedings of the International Conference on Learning Representations, Virtual Event, 3–7 May 2021; OpenReview.net: Amherst, MA, USA, 2021. [Google Scholar]
- Luo, S.; Hu, W. Diffusion Probabilistic Models for 3D Point Cloud Generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Virtual, 20–25 June 2021; Computer Vision Foundation/IEEE: Piscatway, NJ, USA, 2021; pp. 2837–2845. [Google Scholar] [CrossRef]
- Sun, J.; Wang, J.; Nie, W.; Yu, Z.; Mao, Z.; Xiao, C. A Critical Revisit of Adversarial Robustness in 3D Point Cloud Recognition with Diffusion-Driven Purification. In Proceedings of the International Conference on Machine Learning—PMLR, Honolulu, HI, USA, 1 July 2023; Volume 202, pp. 33100–33114. [Google Scholar]
- Zhou, Y.; Tuzel, O. VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; Computer Vision Foundation/IEEE Computer Society: Washington, DC, USA, 2018; pp. 4490–4499. [Google Scholar] [CrossRef]
- Yan, Y.; Mao, Y.; Li, B. SECOND: Sparsely Embedded Convolutional Detection. Sensors 2018, 18, 3337. [Google Scholar] [CrossRef] [PubMed]
- Lang, A.H.; Vora, S.; Caesar, H.; Zhou, L.; Yang, J.; Beijbom, O. PointPillars: Fast Encoders for Object Detection From Point Clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 15–20 June 2019; Computer Vision Foundation/IEEE: Piscataway, NJ, USA, 2019; pp. 12697–12705. [Google Scholar] [CrossRef]
- Shi, S.; Wang, X.; Li, H. PointRCNN: 3D Object Proposal Generation and Detection From Point Cloud. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 15–20 June 2019; Computer Vision Foundation/IEEE: Piscataway, NJ, USA, 2019; pp. 770–779. [Google Scholar] [CrossRef]
- Shi, S.; Guo, C.; Jiang, L.; Wang, Z.; Shi, J.; Wang, X.; Li, H. PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 13–19 June 2020; Computer Vision Foundation/IEEE: Piscataway, NJ, USA; pp. 10526–10535. [Google Scholar] [CrossRef]
- Yin, T.; Zhou, X.; Krähenbühl, P. Center-Based 3D Object Detection and Tracking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Virtual, 20–25 June 2021; Computer Vision Foundation/IEEE: Piscataway, NJ, USA, 2021; pp. 11784–11793. [Google Scholar] [CrossRef]
- Zhou, Z.; Zhao, X.; Wang, Y.; Wang, P.; Foroosh, H. CenterFormer: Center-Based Transformer for 3D Object Detection. In Proceedings of the European Conference on Computer Vision, Tel Aviv, Israel, 23–27 October 2022; Lecture Notes in Computer Science. Springer: Berlin, Germany, 2022; Volume 13698, pp. 496–513. [Google Scholar] [CrossRef]
- Cao, Y.; Xiao, C.; Cyr, B.; Zhou, Y.; Park, W.; Rampazzi, S.; Chen, Q.A.; Fu, K.; Mao, Z.M. Adversarial Sensor Attack on LiDAR-based Perception in Autonomous Driving. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security, London, UK, 6 November 2019; ACM: New York, NY, USA, 2019; pp. 2267–2281. [Google Scholar] [CrossRef]
- Lehner, A.; Gasperini, S.; Marcos-Ramiro, A.; Schmidt, M.; Mahani, M.A.N.; Navab, N.; Busam, B.; Tombari, F. 3D-VField: Adversarial Augmentation of Point Clouds for Domain Generalization in 3D Object Detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18–24 June 2022; IEEE: Piscataway, NJ, USA, 2022; pp. 17274–17283. [Google Scholar] [CrossRef]
- Shin, H.; Kim, D.; Kwon, Y.; Kim, Y. Illusion and Dazzle: Adversarial Optical Channel Exploits Against Lidars for Automotive Applications. In Proceedings of the Cryptographic Hardware and Embedded Systems, Taipei, Taiwan, 25–28 September 2017; Lecture Notes in Computer Science. Springer: Berlin, Germany, 2017; Volume 10529, pp. 445–467. [Google Scholar] [CrossRef]
- Kilic, V.; Hegde, D.; Sindagi, V.; Cooper, A.B.; Foster, M.A.; Patel, V.M. Lidar Light Scattering Augmentation (LISA): Physics-based Simulation of Adverse Weather Conditions for 3D Object Detection. arXiv 2021, arXiv:2107.07004. [Google Scholar]
- Hahner, M.; Sakaridis, C.; Dai, D.; Gool, L.V. Fog Simulation on Real LiDAR Point Clouds for 3D Object Detection in Adverse Weather. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada, 10–17 October 2021; IEEE: Piscataway, NJ, USA, 2021; pp. 15263–15272. [Google Scholar] [CrossRef]
- Bijelic, M.; Gruber, T.; Mannan, F.; Kraus, F.; Ritter, W.; Dietmayer, K.; Heide, F. Seeing Through Fog Without Seeing Fog: Deep Multimodal Sensor Fusion in Unseen Adverse Weather. In Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, 13–19 June 2020; Computer Vision Foundation/IEEE: Piscataway, NJ, USA, 2020; pp. 11679–11689. [Google Scholar] [CrossRef]
- Hahner, M.; Sakaridis, C.; Bijelic, M.; Heide, F.; Yu, F.; Dai, D.; Gool, L.V. LiDAR Snowfall Simulation for Robust 3D Object Detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18–24 June 2022; IEEE: Piscataway, NJ, USA, 2022; pp. 16343–16353. [Google Scholar] [CrossRef]
- Yu, K.; Tao, T.; Xie, H.; Lin, Z.; Liang, T.; Wang, B.; Chen, P.; Hao, D.; Wang, Y.; Liang, X. Benchmarking the Robustness of LiDAR-Camera Fusion for 3D Object Detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Vancouver, BC, Canada, 17–24 June 2023; IEEE: Piscataway, NJ, USA, 2023; pp. 3188–3198. [Google Scholar] [CrossRef]
- Teufel, S.; Gamerdinger, J.; Volk, G.; Gerum, C.; Bringmann, O. Enhancing Robustness of LiDAR-Based Perception in Adverse Weather using Point Cloud Augmentations. In Proceedings of the IEEE Intelligent Vehicles Symposium, Anchorage, AK, USA, 3–7 June 2023; IEEE: Piscataway, NJ, USA, 2023; pp. 1–6. [Google Scholar] [CrossRef]
- Zhou, H.; Chen, K.; Zhang, W.; Fang, H.; Zhou, W.; Yu, N. DUP-Net: Denoiser and Upsampler Network for 3D Adversarial Point Clouds Defense. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea, 27 October–2 November 2019; IEEE: Piscataway, NJ, USA, 2019; pp. 1961–1970. [Google Scholar] [CrossRef]
- Dong, X.; Chen, D.; Zhou, H.; Hua, G.; Zhang, W.; Yu, N. Self-Robust 3D Point Recognition via Gather-Vector Guidance. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 13–19 June 2020; Computer Vision Foundation/IEEE: Piscataway, NJ, USA, 2020; pp. 11513–11521. [Google Scholar] [CrossRef]
- Sun, J.; Cao, Y.; Choy, C.B.; Yu, Z.; Anandkumar, A.; Mao, Z.M.; Xiao, C. Adversarially Robust 3D Point Cloud Recognition Using Self-Supervisions. In Proceedings of the Annual Conference on Neural Information Processing Systems, Virtual, 6–14 December 2021; pp. 15498–15512. [Google Scholar]
- Wu, Z.; Duan, Y.; Wang, H.; Fan, Q.; Guibas, L.J. IF-Defense: 3D Adversarial Point Cloud Defense via Implicit Function based Restoration. arXiv 2020, arXiv:2010.05272. [Google Scholar]
- Li, K.; Zhang, Z.; Zhong, C.; Wang, G. Robust Structured Declarative Classifiers for 3D Point Clouds: Defending Adversarial Attacks with Implicit Gradients. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18–24 June 2022; IEEE: Piscataway, NJ, USA, 2022; pp. 15273–15283. [Google Scholar] [CrossRef]
- Lyu, Z.; Kong, Z.; Xu, X.; Pan, L.; Lin, D. A Conditional Point Diffusion-Refinement Paradigm for 3D Point Cloud Completion. In Proceedings of the International Conference on Learning Representations, Virtual Event, 25–29 April 2022; OpenReview.net: Amherst, MA, USA, 2022. [Google Scholar]
- Zeng, X.; Vahdat, A.; Williams, F.; Gojcic, Z.; Litany, O.; Fidler, S.; Kreis, K. LION: Latent Point Diffusion Models for 3D Shape Generation. In Proceedings of the Annual Conference on Neural Information Processing Systems, New Orleans, LA, USA, 28 November 2022. [Google Scholar]
- Xu, J.; Wang, X.; Cheng, W.; Cao, Y.P.; Shan, Y.; Qie, X.; Gao, S. Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada, June 2023; IEEE: Piscataway, NJ, USA, 2023; pp. 20908–20918. [Google Scholar] [CrossRef]
- Lee, J.; Im, W.; Lee, S.; Yoon, S.E. Diffusion Probabilistic Models for Scene-Scale 3D Categorical Data. arXiv 2023, arXiv:2301.00527. [Google Scholar] [CrossRef]
- Nunes, L.; Marcuzzi, R.; Mersch, B.; Behley, J.; Stachniss, C. Scaling Diffusion Models to Real-World 3D LiDAR Scene Completion. arXiv 2024, arXiv:2403.13470. [Google Scholar] [CrossRef]
- Nie, W.; Guo, B.; Huang, Y.; Xiao, C.; Vahdat, A.; Anandkumar, A. Diffusion Models for Adversarial Purification. In Proceedings of the International Conference on Machine Learning—PMLR, Baltimore, MD, USA, 17–23 July 2022; Volume 162, pp. 16805–16827. [Google Scholar]
- Qi, C.R.; Yi, L.; Su, H.; Guibas, L.J. PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space. In Proceedings of the Annual Conference on Neural Information Processing Systems, Long Beach, CA, USA, 4–9 December 2017; pp. 5099–5108. [Google Scholar]
- Qi, C.R.; Su, H.; Mo, K.; Guibas, L.J. PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; IEEE Computer Society: Washington, DC, USA, 2017; pp. 77–85. [Google Scholar] [CrossRef]
- Lechner, M.; Hasani, R.M.; Grosu, R.; Rus, D.; Henzinger, T.A. Adversarial Training is Not Ready for Robot Learning. In Proceedings of the IEEE International Conference on Robotics and Automation, Xi’an, China, 30 May–5 June 2021; IEEE: Piscataway, NJ, USA, 2021; pp. 4140–4147. [Google Scholar] [CrossRef]
- Geiger, A.; Lenz, P.; Stiller, C.; Urtasun, R. Vision meets robotics: The KITTI dataset. Int. J. Robot. Res. 2013, 32, 1231–1237. [Google Scholar] [CrossRef]
- Kingma, D.P.; Ba, J. Adam: A Method for Stochastic Optimization. In Proceedings of the International Conference on Learning Representations, San Diego, CA, USA, 7–9 May 2015; Bengio, Y., LeCun, Y., Eds.; [Google Scholar]
- Lu, C.; Zhou, Y.; Bao, F.; Chen, J.; Li, C.; Zhu, J. DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps. In Proceedings of the Annual Conference on Neural Information Processing Systems, New Orleans, LA, USA, 28 November–9 December 2022. [Google Scholar]
Corruptions | Adversarial Training | |||
---|---|---|---|---|
None | Snow | Rain | Fog | |
None | 78.4 | 78.1 | 77.5 | 77.2 |
Snow | 36.5 | 39.5 | 34.0 | 33.1 |
Rain | 36.2 | 35.2 | 38.8 | 35.7 |
Fog | 64.3 | 64.1 | 63.8 | 66.3 |
Defense Strategies (Robust AP) | |||||||||
---|---|---|---|---|---|---|---|---|---|
Corruptions | (Standard AP) | Snowfall Simulation [31] | LISA [28] | Fog Simulation [29] | Teufel et al. [33] | DUP-Net [34] | IF-Defense [37] | Ours (Voxel) | Ours (Pillar) |
No Corruption | 78.4 | 78.1 | 77.5 | 77.9 | 78.0 | 54.2 | 59.4 | 78.3 | 78.4 |
Snow | 36.5 | 39.5 | 34.0 | 33.1 | 37.8 | 26.2 | 28.2 | 44.6 | 44.0 |
Rain | 36.2 | 35.2 | 38.8 | 35.7 | 39.6 | 25.3 | 26.2 | 43.9 | 43.1 |
Fog | 64.3 | 64.1 | 63.8 | 66.3 | 64.3 | 43.4 | 47.4 | 71.7 | 71.5 |
Sunlight | 62.3 | 61.4 | 61.7 | 61.5 | 61.7 | 45.0 | 48.7 | 66.8 | 66.4 |
Density | 76.5 | 76.2 | 75.9 | 76.2 | 75.4 | 54.0 | 56.9 | 77.2 | 77.6 |
Cutout | 70.3 | 70.1 | 70.0 | 70.4 | 69.8 | 50.0 | 54.1 | 73.0 | 73.2 |
Crosstalk | 70.9 | 68.7 | 69.3 | 68.8 | 69.0 | 48.9 | 50.5 | 71.6 | 72.5 |
Gaussian | 74.7 | 74.0 | 73.8 | 74.2 | 73.8 | 50.4 | 59.3 | 77.1 | 76.5 |
Uniform | 77.3 | 77.2 | 76.8 | 77.2 | 76.4 | 56.3 | 56.1 | 78.0 | 77.7 |
Impulse | 78.2 | 78.2 | 78.1 | 78.3 | 78.1 | 54.1 | 60.9 | 78.2 | 78.2 |
Moving object | 50.2 | 50.3 | 50.0 | 52.1 | 52.5 | 33.8 | 37.4 | 48.4 | 50.2 |
Local Density | 69.6 | 69.4 | 69.5 | 69.7 | 69.9 | 48.8 | 52.4 | 73.6 | 73.4 |
Local Cutout | 61.8 | 60.6 | 61.6 | 60.5 | 60.4 | 41.9 | 44.3 | 66.2 | 65.7 |
Local Gaussian | 76.7 | 76.7 | 76.4 | 77.0 | 76.5 | 52.8 | 60.7 | 77.7 | 77.6 |
Local Uniform | 78.0 | 78.0 | 77.9 | 78.2 | 78.0 | 54.2 | 59.5 | 78.2 | 77.9 |
Local Impulse | 78.4 | 78.4 | 78.3 | 78.4 | 78.4 | 53.3 | 56.5 | 78.5 | 78.4 |
Shear | 39.6 | 36.9 | 39.0 | 37.8 | 36.6 | 28.7 | 31.2 | 39.9 | 40.4 |
Scale | 70.3 | 70.2 | 69.9 | 70.6 | 69.4 | 47.7 | 51.4 | 73.3 | 73.2 |
Rotation | 72.7 | 72.8 | 72.3 | 72.7 | 72.6 | 50.5 | 57.9 | 73.4 | 72.2 |
Average | 65.5 | 65.2 | 65.1 | 65.2 | 65.3 | 45.5 | 49.5 | 68.0 | 67.9 |
Defense Strategies (Robust AP) | |||||||||
---|---|---|---|---|---|---|---|---|---|
Corruptions | (Standard AP) | Snowfall Simulation [31] | LISA [28] | Fog Simulation [29] | Teufel et al. [33] | DUP-Net [34] | IF-Defense [37] | Ours (Voxel) | Ours (Pillar) |
No Corruption | 80.6 | 80.5 | 80.1 | 79.4 | 80.2 | 55.9 | 58.9 | 80.3 | 80.0 |
Snow | 50.4 | 54.3 | 49.8 | 49.9 | 52.7 | 36.0 | 36.3 | 55.9 | 54.5 |
Rain | 51.3 | 51.0 | 53.4 | 50.6 | 51.9 | 36.4 | 36.8 | 54.2 | 53.8 |
Fog | 72.1 | 71.6 | 71.9 | 72.7 | 72.8 | 51.6 | 54.7 | 73.7 | 73.1 |
Sunlight | 62.8 | 61.6 | 62.1 | 62.0 | 62.2 | 42.5 | 48.3 | 64.3 | 63.6 |
Density | 80.4 | 78.2 | 80.5 | 79.7 | 79.4 | 56.0 | 62.3 | 80.3 | 80.5 |
Cutout | 73.9 | 73.6 | 71.6 | 73.2 | 72.5 | 52.1 | 53.4 | 74.7 | 75.1 |
Crosstalk | 71.5 | 69.8 | 71.2 | 71.5 | 71.3 | 51.6 | 52.0 | 73.5 | 73.7 |
Gaussian | 61.2 | 60.9 | 60.0 | 60.1 | 60.3 | 42.5 | 47.6 | 64.1 | 63.6 |
Uniform | 76.4 | 76.5 | 74.9 | 76.3 | 76.6 | 52.1 | 55.5 | 77.5 | 77.2 |
Impulse | 79.8 | 78.9 | 78.8 | 77.8 | 79.1 | 56.9 | 57.0 | 80.2 | 80.0 |
Moving object | 50.5 | 50.2 | 50.5 | 50.3 | 50.2 | 34.7 | 37.8 | 51.9 | 52.6 |
Local Density | 74.2 | 72.8 | 72.7 | 73.1 | 73.2 | 52.4 | 54.9 | 77.7 | 76.8 |
Local Cutout | 67.9 | 66.2 | 66.6 | 67.4 | 66.7 | 45.8 | 48.9 | 69.9 | 69.7 |
Local Gaussian | 69.8 | 68.7 | 67.5 | 69.2 | 69.0 | 47.5 | 54.3 | 72.4 | 72.1 |
Local Uniform | 77.7 | 77.5 | 76.1 | 76.5 | 77.4 | 53.9 | 58.9 | 79.6 | 79.3 |
Local Impulse | 80.3 | 80.1 | 78.9 | 79.7 | 79.8 | 54.6 | 61.5 | 80.2 | 80.3 |
Shear | 39.8 | 39.7 | 39.8 | 39.1 | 39.5 | 27.0 | 30.7 | 40.9 | 41.4 |
Scale | 71.5 | 71.6 | 70.3 | 71.3 | 71.1 | 49.8 | 54.5 | 74.1 | 73.5 |
Rotation | 75.6 | 75.4 | 75.4 | 74.4 | 75.3 | 51.6 | 54.6 | 77.2 | 77.0 |
Average | 67.7 | 67.3 | 66.9 | 67.1 | 67.4 | 47.1 | 50.5 | 69.6 | 69.3 |
Defense Strategies (Robust AP) | |||||||||
---|---|---|---|---|---|---|---|---|---|
Corruptions | (Standard AP) | Snowfall Simulation [31] | LISA [28] | Fog Simulation [29] | Teufel et al. [33] | DUP-Net [34] | IF-Defense [37] | Ours (Voxel) | Ours (Pillar) |
No Corruption | 84.4 | 84.0 | 83.9 | 83.8 | 84.0 | 58.0 | 65.5 | 84.2 | 84.1 |
Snow | 52.4 | 55.7 | 51.6 | 51.2 | 54.6 | 37.7 | 39.2 | 56.5 | 56.0 |
Rain | 51.6 | 51.3 | 53.1 | 50.8 | 52.8 | 36.0 | 39.6 | 54.1 | 53.7 |
Fog | 79.5 | 79.0 | 79.2 | 80.7 | 80.4 | 54.7 | 57.8 | 82.1 | 81.6 |
Sunlight | 79.9 | 79.4 | 78.2 | 79.1 | 79.7 | 53.6 | 60.6 | 82.0 | 81.3 |
Density | 82.8 | 82.6 | 81.0 | 79.5 | 81.7 | 57.8 | 63.1 | 83.2 | 83.5 |
Cutout | 76.1 | 74.3 | 74.3 | 76.2 | 75.6 | 53.1 | 55.3 | 77.5 | 78.0 |
Crosstalk | 82.3 | 80.4 | 79.6 | 82.4 | 82.6 | 56.9 | 58.7 | 82.6 | 82.8 |
Gaussian | 65.1 | 63.4 | 63.7 | 63.0 | 64.0 | 43.9 | 50.1 | 68.6 | 68.4 |
Uniform | 81.2 | 80.6 | 79.2 | 78.4 | 80.4 | 58.4 | 62.8 | 82.1 | 82.2 |
Impulse | 82.8 | 81.6 | 81.4 | 80.4 | 81.1 | 57.4 | 62.1 | 83.3 | 83.0 |
Moving object | 54.6 | 53.7 | 53.0 | 53.4 | 54.0 | 39.0 | 42.1 | 55.8 | 55.5 |
Local Density | 77.6 | 75.3 | 76.6 | 75.6 | 76.9 | 56.2 | 55.2 | 80.3 | 79.8 |
Local Cutout | 72.3 | 70.6 | 70.7 | 71.3 | 71.4 | 50.4 | 51.9 | 75.8 | 75.9 |
Local Gaussian | 70.4 | 68.5 | 69.5 | 67.9 | 69.4 | 50.3 | 51.4 | 74.1 | 73.4 |
Local Uniform | 82.1 | 80.3 | 80.9 | 79.4 | 80.5 | 58.0 | 63.6 | 83.4 | 83.2 |
Local Impulse | 84.0 | 81.7 | 81.1 | 83.2 | 82.6 | 58.7 | 61.0 | 84.0 | 84.1 |
Shear | 47.7 | 47.0 | 46.7 | 47.4 | 47.0 | 33.0 | 36.2 | 48.6 | 49.3 |
Scale | 76.8 | 75.0 | 75.9 | 74.3 | 75.7 | 54.8 | 55.0 | 78.7 | 78.8 |
Rotation | 79.9 | 77.6 | 77.9 | 79.7 | 79.0 | 55.1 | 62.0 | 80.4 | 80.0 |
Average | 72.6 | 71.5 | 71.2 | 71.3 | 72.1 | 50.8 | 54.1 | 74.4 | 74.2 |
Denoising Recovery Loss | l2-norm | Chamfer Distance | Mean Robust AP | |||
---|---|---|---|---|---|---|
diff | pos | PointPillars | PointRCNN | PV-RCNN | ||
- | - | ✓ | × | 64.1 | 67.4 | 69.7 |
- | - | × | ✓ | 65.9 | 68.3 | 71.3 |
✓ | × | - | - | 65.6 | 68.8 | 72.8 |
× | ✓ | - | - | 66.4 | 68.2 | 71.5 |
✓ | ✓ | - | - | 68.0 | 69.6 | 74.4 |
r for Voxel | r for Pillar | |||||||||
---|---|---|---|---|---|---|---|---|---|---|
0.1 | 0.15 | 0.2 | 0.25 | 0.3 | 0.1 | 0.15 | 0.2 | 0.25 | 0.3 | |
PointPillars | 64.1 | 67.8 | 68 | 65.2 | 58.9 | 68.1 | 67.9 | 64.2 | 60.6 | 52.3 |
PointRCNN | 64.0 | 68.3 | 69.6 | 65.0 | 59.7 | 67.8 | 69.3 | 67.8 | 62.0 | 57.1 |
PV-RCNN | 67.4 | 72.6 | 74.4 | 69.2 | 62.2 | 72.9 | 74.2 | 72.0 | 69.5 | 64.8 |
time consumption | 2.74 | 2.16 | 1.31 | 0.85 | 0.64 | 1.67 | 1.29 | 0.94 | 0.56 | 0.33 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Cai, M.; Wang, X.; Sohel, F.; Lei, H. Diffusion Models-Based Purification for Common Corruptions on Robust 3D Object Detection. Sensors 2024, 24, 5440. https://doi.org/10.3390/s24165440
Cai M, Wang X, Sohel F, Lei H. Diffusion Models-Based Purification for Common Corruptions on Robust 3D Object Detection. Sensors. 2024; 24(16):5440. https://doi.org/10.3390/s24165440
Chicago/Turabian StyleCai, Mumuxin, Xupeng Wang, Ferdous Sohel, and Hang Lei. 2024. "Diffusion Models-Based Purification for Common Corruptions on Robust 3D Object Detection" Sensors 24, no. 16: 5440. https://doi.org/10.3390/s24165440