Abnormality Detection of Blast Furnace Tuyere Based on Knowledge Distillation and a Vision Transformer
Abstract
:1. Introduction
2. Methods
2.1. BDiT
2.2. Spatial Attention Module
2.3. Improved Self-Attention Mechanism
2.4. Loss Function
3. Experimental Results and Analysis
3.1. Dataset Selection and Preprocessing
- (1)
- To reduce computational complexity and prevent size distortion, the resolution of the blast furnace tuyere images was standardized from the original 960 × 576 to 256 × 256;
- (2)
- To address the issue of potential image mirroring caused by factors such as shooting angles or sensor installation methods in blast furnace tuyere images, this study applied horizontal or vertical flipping to the images. By correcting the image mirroring problem and aligning it with the actual scene, the diversity of the data was increased, which helped improve the model’s generalization ability and enabled it to better adapt to images from different tuyere orientations or mirror scenarios;
- (3)
- Considering the presence of particulate matter or pollutants to a certain extent in a blast furnace tuyere, as well as the potential degradation of information transmission in the tuyere camera over time, this study randomly selected 4860 images (810 images per category) from the pool of 24,300 images and added salt-and-pepper noise to them. This random introduction of black and white pixels in the images simulated the loss or degradation of details in blast furnace tuyere images, making them more consistent with the actual conditions observed in tuyere environments.
3.2. Evaluation Metrics and Experimental Settings
3.3. Ablation Experiment
3.3.1. Self-Attention Mechanism and Spatial Attention Module
3.3.2. Distillation Loss
No. | Method | Model Composition | Teacher | Accuracy (%) |
---|---|---|---|---|
(1) | ViT [13] | SA | - | 96.74 |
(2) | ViT(EA) | EA | - | 96.80 |
(3) | ViT(IEA) | IEA | - | 97.33 |
(4) | ViT(IEA&SPA) | IEA+SPA | - | 97.52 |
(5) | ViT(SA&L_DIS) | SA+L_DIS | - | 97.58 |
(6) | ViT(IEA&SPA&L_DIS) | IEA+SPA+L_DIS | - | 97.86 |
(7) | BDiT (IEA&SPA&L_DIS1) | IEA+SPA+L_DIS | VGG-19 | 97.31 |
(8) | BDiT (IEA&SPA&L_DIS2) | IEA+SPA+L_DIS | ResNet-101 | 96.88 |
(9) | BDiT (IEA&SPA&L_DIS3) | IEA+SPA+L_DIS | ResNet-50 | 97.86 |
3.4. Comparative Experiment of Different Methods
4. Conclusions
- (1)
- Reducing the time required for abnormal state detection in blast furnace tuyeres;
- (2)
- Enhancing the classification accuracy of individual state images.
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Wen, L.; Bai, C.; Ouyang, Q.; Chen, F.; Dong, L.; Qiu, G. Recent Research on Supervision Running State of the Tuyere and Raceway in Blast Furnace. J. Chongqing Univ. (Nat. Sci. Ed.) 2005, 8, 39–41. [Google Scholar]
- Zhang, T.; Zhang, X.; Han, T.; Shi, Z.; Guo, Y.; Wang, H. Application of artificial intelligence image recognition technology in blast furnace tuyere monitoring. Metall. Autom. 2021, 45, 58–66. [Google Scholar]
- Zhang, T.; Zhang, J.; Peng, G.; Wang, H. Automated Machine Learning for Steel Production: A Case Study of TPOT for Material Mechanical Property Prediction. In Proceedings of the 2022 IEEE International Conference on e-Business Engineering (ICEBE), Bournemouth, UK, 14–16 October 2022; IEEE: New York, NY, USA, 2022. [Google Scholar]
- Choi, Y.; Kwun, H.; Kim, D.; Lee, E.; Bae, H. Method of predictive maintenance for induction furnace based on neural network. In Proceedings of the 2020 IEEE International Conference on Big Data and Smart Computing (BigComp), Busan, Republic of Korea, 19–22 February 2020; IEEE: New York, NY, USA, 2020. [Google Scholar]
- Zhao, X.; Amp, H.E. Current Research on Characteristics of Tuyere Raceway of Blast Furnace. Gansu Metall. 2015, 37, 35–39. [Google Scholar]
- Liu, J.; Gao, Y.; Yang, H.; Li, F. Innovation Research of Large Blast Furnace Tuyere Equipment Maintenance Technology. Constr. Technol. 2018, 47, 498–500. [Google Scholar]
- Qin, T.; Ma, L.; Chen, C.; Wang, Y. Research on blast furnace air outlet coal injection flow detection method. Ind. Metrol. 2015, 25, 36–39+66. [Google Scholar] [CrossRef]
- Puttinger, S.; Stocker, H. Toward a Better Understanding of Blast Furnace Raceway Blockages. Steel Res. Int. 2020, 91, 202000227. [Google Scholar] [CrossRef]
- Born, S.; Babich, A.; van der Stel, J.; Ho, H.T.; Sert, D.; Ansseau, O.; Plancq, C.; Pierret, J.-C.; Geyer, R.; Senk, D.; et al. Char formation by coal injection and its behavior in the blast furnace. Steel Res. Int. 2020, 91, 2000038. [Google Scholar] [CrossRef]
- Fu, J.; Zheng, H.; Mei, T. Look Closer to See Better: Recurrent Attention Convolutional Neural Network for Fine-Grained Image Recognition. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017; pp. 4476–4484. [Google Scholar]
- Zheng, H.; Fu, J.; Mei, T.; Luo, J. Learning Multi-attention Convolutional Neural Network for Fine-Grained Image Recognition. In Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22–29 October 2017; IEEE: New York, NY, USA, 2017. [Google Scholar]
- Zheng, H.; Fu, J.; Zha, Z.J.; Luo, J.; Mei, T. Learning Rich Part Hierarchies with Progressive Attention Networks for Fine-Grained Image Recognition. IEEE Trans Image Process. 2019, 29, 476–488. [Google Scholar] [CrossRef]
- Wei, X.-S.; Song, Y.-Z.; Mac Aodha, O.; Wu, J.; Peng, Y.; Tang, J.; Yang, J.; Belongie, S. Fine-Grained Image Analysis with Deep Learning: A Survey. IEEE Trans. Pattern Anal. Mach. Intell. 2021, 44, 8927–8948. [Google Scholar] [CrossRef]
- Wang, Y.; Morariu, V.I.; Davis, L.S. Learning a Discriminative Filter Bank within a CNN for Fine-Grained Recognition. In Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA, 18–23 June 2018; IEEE: New York, NY, USA, 2018. [Google Scholar]
- Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; et al. An Image is Worth 16 × 16 Words: Transformers for Image Recognition at Scale. Int. Conf. Learn. Represent. arXiv 2021, arXiv:2010.11929. [Google Scholar]
- Guo, M.-H.; Liu, Z.-N.; Mu, T.-J.; Hu, S.-M. Beyond Self-Attention: External Attention Using Two Linear Layers for Visual Tasks. IEEE Trans. Pattern Anal. Mach. Intell. 2022, 45, 5436–5447. [Google Scholar] [CrossRef] [PubMed]
- He, J.; Chen, J.-N.; Liu, S.; Kortylewski, A.; Yang, C.; Bai, Y.; Wang, C. TransFG: A Transformer Architecture for Fine-Grained Recognition. In Proceedings of the AAAI Conference on Artificial Intelligence, Washington, DC, USA, 7–14 February 2022; Volume 36, pp. 852–860. [Google Scholar] [CrossRef]
- Conde, M.V.; Turgutlu, K. Exploring Vision Transformers for Fine-grained Classification. arXiv 2021, arXiv:2106.10587. [Google Scholar]
- Li, Y.; Zhang, K.; Cao, J.; Timofte, R.; Van Gool, L. Localvit: Bringing locality to vision transformers. arXiv 2021, arXiv:2104.0570. [Google Scholar]
- Zhang, Q.; Yang, Y.-B. Rest: An efficient transformer for visual recognition. Adv. Neural Inf. Process. Syst. 2021, 34, 15475–15485. [Google Scholar]
- Hinton, G.; Vinyals, O.; Dean, J. Distilling the Knowledge in a Neural Network. Comput. Sci. 2015, 14, 38–39. [Google Scholar]
- Touvron, H.; Cord, M.; Douze, M.; Massa, F.; Sablayrolles, A.; Jégou, H. Training data-efficient image transformers & distillation through attention. arXiv 2020, arXiv:2012.12877. [Google Scholar]
- Zheng, Z.; Xi, P. Self-guidance: Improve deep neural network generalization via knowledge distillation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA, 3–8 January 2022. [Google Scholar]
- Wang, R.; Li, Z.; Yang, L.; Li, Y.; Zhang, H.; Song, C.; Jiang, M.; Ye, X.; Hu, K. Application of Efficient Channel Attention Residual Mechanism in Blast Furnace Tuyere Image Anomaly Detection. Appl. Sci. 2022, 12, 7823. [Google Scholar] [CrossRef]
- Song, C.; Li, Z.; Li, Y.; Zhang, H.; Jiang, M.; Hu, K.; Wang, R. Research on Blast Furnace Tuyere Image Anomaly Detection, Based on the Local Channel Attention Residual Mechanism. Appl. Sci. 2023, 13, 802. [Google Scholar] [CrossRef]
- Park, J.; Woo, S.; Lee, J.-Y.; Kweon, I.S. BAM: Bottleneck Attention Module. arXiv 2018, arXiv:1807.06514. [Google Scholar]
- Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, Ł. Attention is all you need. Adv. Neural Inf. Process. Syst. 2017, 30, 6000–6010. [Google Scholar]
- Khosla, A.; Jayadevaprakash, N.; Yao, B.; Li, F.F. Novel dataset for fine-grained image categorization: Stanford dogs. In Proceedings of the CVPR Workshop on Fine-Grained Visual Categorization (FGVC), Providence, RI, USA, 21–23 June 2011; pp. 1–2. [Google Scholar]
- Wah, C.; Branson, S.; Welinder, P.; Perona, P.; Belongie, S. The Caltech-Ucsd Birds-200-2011 Dataset; CNS-TR-2011-001; California Institute of Technology: Pasadena, CA, USA, 2011. [Google Scholar]
- Wen, L.; Li, X.; Li, X.; Gao, L. A new transfer learning based on VGG-19 network for fault diagnosis. In Proceedings of the 2019 IEEE 23rd International Conference on Computer Supported Cooperative Work in Design (CSCWD), Porto, Portugal, 6–8 May 2019; IEEE: New York, NY, USA, 2019. [Google Scholar]
- Zhang, Q. A novel ResNet101 model based on dense dilated convolution for image classification. SN Appl. Sci. 2021, 4, 9. [Google Scholar] [CrossRef]
- Gan, G.; Xiao, X.; Jiang, C.; Ye, Y.; He, Y.; Xu, Y.; Luo, C. Strawberry Disease and Pest Identification and Control Based on SE-ResNeXt50 Model. In Proceedings of the 2022 3rd International Conference on Computer Vision, Image and Deep Learning & International Conference on Computer Engineering and Applications (CVIDL & ICCEA), Changchun, China, 20–22 May 2022; IEEE: New York, NY, USA, 2022. [Google Scholar]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016. [Google Scholar]
- Aftab, A.; Morsali, A.; Ghaemmaghami, S.; Champagne, B. LIGHT-SERNET: A lightweight fully convolutional neural network for speech emotion recognition. In Proceedings of the ICASSP 2022–2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Singapore, 22–27 May 2022; IEEE: New York, NY, USA, 2022. [Google Scholar]
- Zhou, T.; Zhao, Y.; Wu, J. Resnext and res2net structures for speaker verification. In Proceedings of the 2021 IEEE Spoken Language Technology Workshop (SLT), Shenzhen, China, 19–22 January 2021; IEEE: New York, NY, USA, 2021. [Google Scholar]
- Ding, X.; Zhang, X.; Ma, N.; Han, J.; Ding, G.; Sun, J. Repvgg: Making vgg-style convnets great again. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 20–25 June 2021. [Google Scholar]
Datasets | Category | Total | Train | Test |
---|---|---|---|---|
Tuyere-Data | 6 | 29,160 | 23,328 | 5832 |
Stanford Dogs | 120 | 20,580 | 16,464 | 4116 |
CUB-200-2011 | 200 | 11,788 | 9431 | 2357 |
Model | Accuracy (%) | |
---|---|---|
CUB | Dogs | |
ESERNet [31] | 60.72 | 73.20 |
LSERNet [32] | 63.57 | 74.39 |
ResNet101 [26] | 72.93 | 84.35 |
ResNet50 [27,28] | 74.58 | 87.16 |
repVGG [31] | 69.17 | 85.81 |
ViT [13] | 77.42 | 88.27 |
DeiT [18] | 76.06 | 91.32 |
TransFG [15] | 80.43 | 90.24 |
BDiT | 77.65 | 90.31 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Song, C.; Zhang, H.; Wang, Y.; Wang, Y.; Hu, K. Abnormality Detection of Blast Furnace Tuyere Based on Knowledge Distillation and a Vision Transformer. Appl. Sci. 2023, 13, 10398. https://doi.org/10.3390/app131810398
Song C, Zhang H, Wang Y, Wang Y, Hu K. Abnormality Detection of Blast Furnace Tuyere Based on Knowledge Distillation and a Vision Transformer. Applied Sciences. 2023; 13(18):10398. https://doi.org/10.3390/app131810398
Chicago/Turabian StyleSong, Chuanwang, Hao Zhang, Yuanjun Wang, Yuhui Wang, and Keyong Hu. 2023. "Abnormality Detection of Blast Furnace Tuyere Based on Knowledge Distillation and a Vision Transformer" Applied Sciences 13, no. 18: 10398. https://doi.org/10.3390/app131810398