The Effects of AI-Driven Face Restoration on Forensic Face Recognition
Abstract
:1. Introduction
- Improve image quality with AI-powered image repair technology in the field of forensic human image identification.
- The quantitative effect of the quality of the repaired face image on the recognition performance of the recognition system is analyzed.
- To explore the difference in recognition effect of repaired face images in different recognition systems.
2. Materials and Methods
2.1. Experimental Material
2.2. Experimental Method
3. Related Work
4. Results
4.1. The Impact of Blurriness Level on the Recognition Performance of Face Recognition Systems
4.2. The Influence of Restoration Methods on the Recognition Performance of Face Recognition Systems
4.2.1. Comparison of Four Different Restoration Methods for Blurriness Level 3
4.2.2. Comparison of Four Different Restoration Methods for Blurriness Level 5
4.3. The Impact of Blur Level and Restoration Method on the Recognition Performance of the Face Recognition System
4.3.1. Performing Pairwise Comparisons between Blur Level and Restoration Method in Baidu
4.3.2. Performing Pairwise Comparisons between Blur Level and Restoration Method in Ali Cloud
5. Discussion
6. Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Yan, C.; Meng, L.; Li, L.; Zhang, J.; Wang, Z.; Yin, J.; Zhang, J.; Sun, Y.; Zheng, B. Age-invariant face recognition by multi-feature fusionand decomposition with self-attention. ACM Trans. Multimed. Comput. Commun. Appl. (TOMM) 2022, 18, 1–18. [Google Scholar] [CrossRef]
- Abudarham, N.; Shkiller, L.; Yovel, G. Critical features for face recognition. Cognition 2019, 182, 73–83. [Google Scholar] [CrossRef] [PubMed]
- Zeng, J.; Zhu, H.; Shi, S.; Qiu, X. Face image quality quantitative assessment for forensic identification of human images. In Proceedings of the 2018 IEEE International Conference on Progress in Informatics and Computing (PIC), Suzhou, China, 14–16 December 2018; pp. 113–116. [Google Scholar]
- Dai, Y.; Sun, K.; Huang, W.; Zhang, D.; Dai, G. Attention-based hierarchical pyramid feature fusion structure for efficient face recognition. IET Image Process. 2023, 17, 2399–2409. [Google Scholar] [CrossRef]
- Miranda, G.E.; de Freitas, S.G.; de Abreu Maia, L.V.; Melani, R.F.H. An unusual method of forensic human identification: Use of selfie photographs. Forensic Sci. Int. 2016, 263, e14–e17. [Google Scholar] [CrossRef] [PubMed]
- Zeng, J.; Qiu, X.; Shi, S. Image processing effects on the deep face recognition system. Math. Biosci. Eng. 2021, 18, 1187–1200. [Google Scholar] [CrossRef] [PubMed]
- Phillips, P.J.; White, D.; O’Toole, A.; Norell, K. Human Factors in Forensic Face Identification. In Handbook of Biometrics for Forensic Science; Springer: Berlin, Germany, 2017. [Google Scholar]
- Valentine, T.; Davis, J.P. Forensic facial identification: A practical guide to best practice. In Forensic Facial Identification: Theory and Practice of Identification from Eyewitnesses, Composites and CCTV; Blackwell Pub: Beach Haven, NJ, USA, 2015; pp. 323–347. [Google Scholar]
- Velusamy, S.; Parihar, R.; Kini, R.; Rege, A. FabSoften: Face beautification via dynamic skin smoothing, guided feathering, and texture restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA, USA, 13–19 June 2020; pp. 530–531. [Google Scholar]
- Phillips, P.J.; Yates, A.N.; Hu, Y.; Hahn, C.A.; Noyes, E.; Jackson, K.; Cavazos, J.G.; Jeckeln, G.; Ranjan, R.; Sankaranarayanan, S. Face recognition accuracy of forensic examiners, superrecognizers, and face recognition algorithms. Proc. Natl. Acad. Sci. USA 2018, 115, 6171–6176. [Google Scholar] [CrossRef]
- Carragher, D.J.; Hancock, P.J. Simulated automated facial recognition systems as decision-aids in forensic face matching tasks. J. Exp. Psychol. Gen. 2023, 152, 1286. [Google Scholar] [CrossRef] [PubMed]
- Koo, J.H.; Cho, S.W.; Baek, N.R.; Park, K.R. Face and body-based human recognition by GAN-based blur restoration. Sensors 2020, 20, 5229. [Google Scholar] [CrossRef] [PubMed]
- Dagnes, N.; Vezzetti, E.; Marcolin, F.; Tornincasa, S. Occlusion detection and restoration techniques for 3D face recognition: A literature review. Mach. Vis. Appl. 2018, 29, 789–813. [Google Scholar] [CrossRef]
- He, M.; Zhang, J.; Shan, S.; Chen, X. Enhancing face recognition with self-supervised 3d reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18–24 June 2022; pp. 4062–4071. [Google Scholar]
- Merkle, J.; Rathgeb, C.; Tams, B.; Lou, D.-P.; Dörsch, A.; Drozdowski, P. State of the art of quality assessment of facial images. arXiv 2022, arXiv:2211.08030. [Google Scholar]
- Fan, L.; Zhang, F.; Fan, H.; Zhang, C. Brief review of image denoising techniques. Vis. Comput. Ind. Biomed. Art 2019, 2, 1–12. [Google Scholar] [CrossRef] [PubMed]
- Wang, Y.; Song, W.; Fortino, G.; Qi, L.-Z.; Zhang, W.; Liotta, A. An experimental-based review of image enhancement and image restoration methods for underwater imaging. IEEE Access 2019, 7, 140233–140251. [Google Scholar] [CrossRef]
- Dong, W.; Zhang, L.; Shi, G.; Li, X. Nonlocally centralized sparse representation for image restoration. IEEE Trans. Image Process. 2012, 22, 1620–1630. [Google Scholar] [CrossRef] [PubMed]
- Zoran, D.; Weiss, Y. From learning models of natural image patches to whole image restoration. In Proceedings of the 2011 International Conference on Computer Vision, Barcelona, Spain, 6–13 November 2011; pp. 479–486. [Google Scholar]
- Mao, X.; Shen, C.; Yang, Y.-B. Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. Adv. Neural Inf. Process. Syst. 2016, 29. [Google Scholar] [CrossRef]
- Ali, A.M.; Benjdira, B.; Koubaa, A.; El-Shafai, W.; Khan, Z.; Boulila, W. Vision transformers in image restoration: A survey. Sensors 2023, 23, 2385. [Google Scholar] [CrossRef] [PubMed]
- Wang, Y.; Hu, Y.; Zhang, J. Panini-Net: GAN prior based degradation-aware feature interpolation for face restoration. Proc. AAAI Conf. Artif. Intell. 2022, 36, 2576–2584. [Google Scholar] [CrossRef]
- Xu, W.; Zhu, Q.; Qi, N.; Chen, D. Deep sparse representation based image restoration with denoising prior. IEEE Trans. Circuits Syst. Video Technol. 2022, 32, 6530–6542. [Google Scholar] [CrossRef]
- Thakur, R.S.; Yadav, R.N.; Gupta, L. State-of-art analysis of image denoising methods using convolutional neural networks. IET Image Process. 2019, 13, 2367–2380. [Google Scholar] [CrossRef]
- Liu, J.; Huang, T.-Z.; Selesnick, I.W.; Lv, X.-G.; Chen, P.-Y. Image restoration using total variation with overlapping group sparsity. Inf. Sci. 2015, 295, 232–246. [Google Scholar] [CrossRef]
- Shen, H.; Peng, L.; Yue, L.; Yuan, Q.; Zhang, L. Adaptive norm selection for regularized image restoration and super-resolution. IEEE Trans. Cybern. 2015, 46, 1388–1399. [Google Scholar] [CrossRef]
- Yue, L.; Shen, H.; Li, J.; Yuan, Q.; Zhang, H.; Zhang, L. Image super-resolution: The techniques, applications, and future. Signal Process. 2016, 128, 389–408. [Google Scholar] [CrossRef]
- An, F.; Wang, J. Image super-resolution reconstruction algorithm based on significant network connection-collaborative migration structure. Digit. Signal Process. 2022, 127, 103566. [Google Scholar] [CrossRef]
- Chua, L.O.; Roska, T. The CNN paradigm. IEEE Trans. Circuits Syst. I Fundam. Theory Appl. 1993, 40, 147–156. [Google Scholar] [CrossRef]
- Bhatt, D.; Patel, C.; Talsania, H.; Patel, J.; Vaghela, R.; Pandya, S.; Modi, K.; Ghayvat, H. CNN variants for computer vision: History, architecture, application, challenges and future scope. Electronics 2021, 10, 2470. [Google Scholar] [CrossRef]
- Zhang, F.; Cai, N.; Wu, J.; Cen, G.; Wang, H.; Chen, X. Image denoising method based on a deep convolution neural network. IET Image Process. 2018, 12, 485–493. [Google Scholar] [CrossRef]
- Yin, W.; Kann, K.; Yu, M.; Schütze, H. Comparative study of CNN and RNN for natural language processing. arXiv 2017, arXiv:1702.01923. [Google Scholar]
- Zhang, Y.; Gan, Z.; Fan, K.; Chen, Z.; Henao, R.; Shen, D.; Carin, L. Adversarial feature matching for text generation. In Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia, 6–11 August 2017; pp. 4006–4015. [Google Scholar]
- Chen, J.; Chen, J.; Chao, H.; Yang, M. Image blind denoising with generative adversarial network based noise modeling. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 3155–3164. [Google Scholar]
- Zhu, J.; Yang, H.; Liu, N.; Kim, M.; Zhang, W.; Yang, M.-H. Online multi-object tracking with dual matching attention networks. In Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, 8–14 September 2018; pp. 366–382. [Google Scholar]
- Cao, Y.-J.; Jia, L.-L.; Chen, Y.-X.; Lin, N.; Yang, C.; Zhang, B.; Liu, Z.; Li, X.-X.; Dai, H.-H. Recent advances of generative adversarial networks in computer vision. IEEE Access 2018, 7, 14985–15006. [Google Scholar] [CrossRef]
- Ullah, I.; Mahmoud, Q.H. Design and development of RNN anomaly detection model for IoT networks. IEEE Access 2022, 10, 62722–62750. [Google Scholar] [CrossRef]
- Mikolov, T.; Kombrink, S.; Burget, L.; Černocký, J.; Khudanpur, S. Extensions of recurrent neural network language model. In Proceedings of the 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Prague, Czech Republic, 22–27 May 2011; pp. 5528–5531. [Google Scholar]
- Liang, L.; Lin, L.; Jin, L.; Xie, D.; Li, M. SCUT-FBP5500: A diverse benchmark dataset for multi-paradigm facial beauty prediction. In Proceedings of the 2018 24th International Conference on Pattern Recognition (ICPR), Beijing, China, 20–24 August 2018; pp. 1598–1603. [Google Scholar]
- Bougourzi, F.; Dornaika, F.; Taleb-Ahmed, A. Deep learning based face beauty prediction via dynamic robust losses and ensemble regression. Knowl. Based Syst. 2022, 242, 108246. [Google Scholar] [CrossRef]
- Saeed, J.N.; Abdulazeez, A.M.; Ibrahim, D.A. FIAC-Net: Facial image attractiveness classification based on light deep convolutional neural network. In Proceedings of the 2022 Second International Conference on Computer Science, Engineering and Applications (ICCSEA), Gunupur, India, 8 September 2022; pp. 1–6. [Google Scholar]
- Hill, J. [Official] HitPaw Photo Enhancer—One Click AI Photo Quality Enhancer. Available online: https://www.hitpaw.com/photo-enhancer.html (accessed on 1 December 2023).
- Labs, T. Topaz Photo AI—Maximize Image Quality with AI. Available online: https://www.topazlabs.com/topaz-photo-ai (accessed on 1 December 2023).
- SMS-iT. Image Format Converter—Image Format Conversion—Quick Image Converter Online. Available online: https://www.xunjietupian.com (accessed on 1 December 2023).
- yangtao9009. GitHub—Yangxy/GPEN. Available online: https://github.com/yangxy/GPEN (accessed on 1 December 2023).
- Baidu. Facial Recognition Cloud Service. Available online: https://cloud.baidu.com/product/face (accessed on 1 December 2023).
- Aliyun. Human Face-Alibaba Cloud Visual Intelligence Open Platform. Available online: https://vision.aliyun.com/facebody (accessed on 1 December 2023).
- Co, A.T. ArcSoft Vision Open Platform. Available online: https://ai.arcsoft.com.cn/product/arcface.html (accessed on 1 December 2023).
Recognition System | MB1 (n = 100) M ± SD | MB2 (n = 100) M ± SD | t | p |
---|---|---|---|---|
Baidu | 98.41 ± 0.52 | 95.32 ± 1.43 | 20.29 | 0.000 |
Ali Cloud | 96.28 ± 1.25 | 89.84 ± 2.04 | 26.93 | 0.000 |
ArcSoft | 99.82 ± 0.63 | 92.87 ± 3.64 | 18.84 | 0.000 |
I | J | Mean Difference (I–J) | Standard Error | Significance |
---|---|---|---|---|
RST 1 | RST 2 | −1.061 * | 0.323 | 0.001 |
RST3 | 0.926 * | 0.323 | 0.004 | |
RST4 | 1.214 * | 0.323 | 0.000 | |
RST 2 | RST 3 | 1.987 * | 0.323 | 0.000 |
RST 4 | 2.275 * | 0.323 | 0.000 | |
RST 3 | RST 4 | 0.287 | 0.323 | 0.374 |
I | J | Mean Difference (I–J) | Standard Error | Significance |
---|---|---|---|---|
RST 1 | RST 2 | −2.685 * | 0.495 | 0.000 |
RST 3 | −0.660 | 0.495 | 0.183 | |
RST 4 | 1.115 * | 0.495 | 0.025 | |
RST 2 | RST 3 | 2.025 * | 0.495 | 0.000 |
RST 4 | 3.800 * | 0.495 | 0.000 | |
RST 3 | RST 4 | 1.775 * | 0.495 | 0.000 |
Restoration Method | Blur Level | |
---|---|---|
B1 | B2 | |
RST 1 | 96.91 ± 1.00 bB | 85.64 ± 7.27 bA |
RST 2 | 97.35 ± 0.86 cB | 89.07 ± 6.02 aA |
RST 3 | 96.44 ± 1.24 aB | 85.93 ± 6.72 bA |
RST 4 | 96.37 ± 2.4 aB | 83.36 ± 7.02 cA |
Restoration Method | Blur Level | |
---|---|---|
B1 | B2 | |
RST 1 | 93.49 ± 1.93 bB | 83.49 ± 3.88 bA |
RST 2 | 94.55 ± 1.79 cB | 86.18 ± 3.09 cA |
RST 3 | 92.56 ± 2.16 aB | 84.16 ± 2.99 bA |
RST 4 | 92.28 ± 3.05 aB | 82.38 ± 3.94 aA |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Yang, M.; Li, S.; Zeng, J. The Effects of AI-Driven Face Restoration on Forensic Face Recognition. Appl. Sci. 2024, 14, 3783. https://doi.org/10.3390/app14093783
Yang M, Li S, Zeng J. The Effects of AI-Driven Face Restoration on Forensic Face Recognition. Applied Sciences. 2024; 14(9):3783. https://doi.org/10.3390/app14093783
Chicago/Turabian StyleYang, Mengxuan, Shengnan Li, and Jinhua Zeng. 2024. "The Effects of AI-Driven Face Restoration on Forensic Face Recognition" Applied Sciences 14, no. 9: 3783. https://doi.org/10.3390/app14093783