Pointer Meter Reading Method Based on YOLOv8 and Improved LinkNet
Abstract
:1. Introduction
- (1)
- In order to solve the problem of slow recognition speed in pointer meter reading methods, we adopted the faster LinkNet semantic segmentation model and improved it. Specifically, we have reduced the model’s parameters and computational load by employing partial convolutions and reducing the number of encoding–decoding blocks, which makes it easier to deploy. Meanwhile, attention modules and feature fusion modules were introduced to ensure the accuracy of the segmentation.
- (2)
- In the subsequent reading process, we propose a rotation correction network based on ResNet18. This network is capable of correcting the rotation of segmented ruler and pointer images to meet the angle requirements for polar coordinate transformation.
- (3)
- Extensive experimental validation has been carried out in environments with different angles and perspectives to demonstrate the feasibility of the proposed method.
2. Methods
2.1. YOLOv8-Based Dial Detection
2.2. Segmentation of the Meter Pointer and Scale Based on Improved LinkNet
2.2.1. PConv Module
2.2.2. CBAM Module
2.2.3. AFF Module
2.3. Rotation Correction
2.4. Reading Methods
3. Experimental Section
3.1. Experimental Configuration
3.2. Indicators for the Assessment of the Model
3.3. Ablation Experiments
3.4. Comparative Experiments
3.5. Analysis of Reading Methods
3.6. Comparison of Different Reading Methods
4. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Gao, J.W.; Xie, H.T.; Zuo, L.; Zhang, C.H. A robust pointer meter reading recognition method for substation inspection robot. In Proceedings of the 2017 International Conference on Robotics and Automation Sciences (ICRAS), Hong Kong, China, 26–29 August 2017; IEEE: Piscataway, NJ, USA, 2017; pp. 43–47. [Google Scholar]
- Mai, X.; Li, W.; Huang, Y.; Yang, Y. An automatic meter reading method based on one-dimensional measuring curve mapping. In Proceedings of the 2018 IEEE International Conference of Intelligent Robotic and Control Engineering (IRCE), Lanzhou, China, 24–27 August 2018; IEEE: Piscataway, NJ, USA, 2018; pp. 69–73. [Google Scholar]
- Hung, M.H.; Hsieh, C.H. Automatic pointer meter reading based on machine vision. In Proceedings of the 2019 IEEE 4th International Conference on Image, Vision and Computing (ICIVC), Xiamen, China, 5–7 July 2019; IEEE: Piscataway, NJ, USA, 2019; pp. 32–35. [Google Scholar]
- Sablatnig, R.; Hansen, C. Machine vision for automatic calibration of analog display instruments. In Proceedings of the Machine Vision Applications in Industrial Inspection III; SPIE: Bellingham, WA, USA, 1995; Volume 2423, pp. 356–366. [Google Scholar]
- Zheng, C.; Wang, S.; Zhang, Y.; Zhang, P.; Zhao, Y. A robust and automatic recognition system of analog instruments in power system by using computer vision. Measurement 2016, 92, 413–420. [Google Scholar] [CrossRef]
- Jiale, H.; En, L.; Bingjie, T.; Ming, L. Reading recognition method of analog measuring instruments based on improved hough transform. In Proceedings of the IEEE 2011 10th International Conference on Electronic Measurement & Instruments, Chengdu, China, 16–19 August 2011; IEEE: Piscataway, NJ, USA, 2011; Volume 3, pp. 337–340. [Google Scholar]
- Zhang, L.; Fang, B.; Zhao, X.; Zhang, H.S. Pointer-type meter automatic reading from complex environment based on visual saliency. In Proceedings of the 2016 International Conference on Wavelet Analysis and Pattern Recognition (ICWAPR), Jeju Island, Republic of Korea, 10–13 July 2016; IEEE: Piscataway, NJ, USA, 2016; pp. 264–269. [Google Scholar]
- Deng, G.; Huang, T.; Lin, B.; Liu, H.; Yang, R.; Jing, W. Automatic meter reading from UAV inspection photos in the substation by combining YOLOv5s and DeepLabv3+. Sensors 2022, 22, 7090. [Google Scholar] [CrossRef] [PubMed]
- Hou, L.; Wang, S.; Sun, X.; Mao, G. A pointer meter reading recognition method based on YOLOX and semantic segmentation technology. Measurement 2023, 218, 113241. [Google Scholar] [CrossRef]
- Dai, Y.; Huang, W.; Zhang, D.; Dai, G. Research and Application of Pointer Meter Reading Algorithm Based on Attention U 2-Net. In Proceedings of the 2022 IEEE 2nd International Conference on Data Science and Computer Application (ICDSCA), Dalian, China, 28–30 October 2022; IEEE: Piscataway, NJ, USA, 2022; pp. 129–133. [Google Scholar]
- Zhou, D.; Yang, Y.; Zhu, J.; Wang, K. Intelligent reading recognition method of a pointer meter based on deep learning in a real environment. Meas. Sci. Technol. 2022, 33, 055021. [Google Scholar] [CrossRef]
- Zhang, C.; Shi, L.; Zhang, D.; Ke, T.; Li, J. Pointer meter recognition method based on Yolov7 and hough transform. Appl. Sci. 2023, 13, 8722. [Google Scholar] [CrossRef]
- Huo, Y.; Bai, H.; Sun, L.; Fang, Y. Reading recognition of pointer meters based on an improved UNet++ network. Meas. Sci. Technol. 2023, 35, 035009. [Google Scholar] [CrossRef]
- Chen, L.; Wu, X.; Sun, C.; Zou, T.; Meng, K.; Lou, P. An intelligent vision recognition method based on deep learning for pointer meters. Meas. Sci. Technol. 2023, 34, 055410. [Google Scholar] [CrossRef]
- Peng, J.; Xu, M.; Yan, Y. Automatic recognition of pointer meter reading based on Yolov4 and improved U-net algorithm. In Proceedings of the 2021 IEEE International Conference on Electronic Technology, Communication and Information (ICETCI), Xi’an, China, 18–20 August 2021; IEEE: Piscataway, NJ, USA, 2021; pp. 52–57. [Google Scholar]
- Chaurasia, A.; Culurciello, E. Linknet: Exploiting encoder representations for efficient semantic segmentation. In Proceedings of the 2017 IEEE Visual Communications and Image Processing (VCIP), St. Petersburg, FL, USA, 10–13 December 2017; IEEE: Piscataway, NJ, USA, 2017; pp. 1–4. [Google Scholar]
- Chen, J.; Kao, S.h.; He, H.; Zhuo, W.; Wen, S.; Lee, C.H.; Chan, S.H.G. Run, Don’t walk: Chasing higher FLOPS for faster neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada, 17–24 June 2023; pp. 12021–12031. [Google Scholar]
- Woo, S.; Park, J.; Lee, J.Y.; Kweon, I.S. Cbam: Convolutional block attention module. In Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, 8–14 September 2018; pp. 3–19. [Google Scholar]
- Dai, Y.; Gieseke, F.; Oehmcke, S.; Wu, Y.; Barnard, K. Attentional feature fusion. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Virtual, 5–9 January 2021; pp. 3560–3569. [Google Scholar]
- Ronneberger, O.; Fischer, P.; Brox, T. U-net: Convolutional networks for biomedical image segmentation. In Proceedings of the Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, 5–9 October 2015; Proceedings, Part III 18. Springer: Berlin/Heidelberg, Germany, 2015; pp. 234–241. [Google Scholar]
- Badrinarayanan, V.; Kendall, A.; Cipolla, R. Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 2017, 39, 2481–2495. [Google Scholar] [CrossRef] [PubMed]
- Chen, L.C.; Zhu, Y.; Papandreou, G.; Schroff, F.; Adam, H. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, 8–14 September 2018; pp. 801–818. [Google Scholar]
- Yu, C.; Wang, J.; Peng, C.; Gao, C.; Yu, G.; Sang, N. Bisenet: Bilateral segmentation network for real-time semantic segmentation. In Proceedings of the European conference on Computer Vision (ECCV), Munich, Germany, 8–14 September 2018; pp. 325–341. [Google Scholar]
- Xu, G.; Li, J.; Gao, G.; Lu, H.; Yang, J.; Yue, D. Lightweight real-time semantic segmentation network with efficient transformer and CNN. IEEE Trans. Intell. Transp. Syst. 2023, 24, 15897–15906. [Google Scholar] [CrossRef]
- Yi, Q.; Dai, G.; Shi, M.; Huang, Z.; Luo, A. Elanet: Effective lightweight attention-guided network for real-time semantic segmentation. Neural Process. Lett. 2023, 55, 6425–6442. [Google Scholar] [CrossRef]
- Paszke, A.; Chaurasia, A.; Kim, S.; Culurciello, E. Enet: A deep neural network architecture for real-time semantic segmentation. arXiv 2016, arXiv:1606.02147. [Google Scholar]
Model | PC (p = 8) | PC (p = 4) | PC (p = 2) | CBAM | AFF | REDB | mIoU/% | Paras/M | FLOPs/G | FPS |
---|---|---|---|---|---|---|---|---|---|---|
LinkNet | 87.86 | 11.54 | 12.14 | 231 | ||||||
1 | ✓ | 87.72 | 6.35 | 8.15 | 256 | |||||
2 | ✓ | ✓ | 86.86 | 1.61 | 6.80 | 304 | ||||
3 | ✓ | ✓ | ✓ | 87.83 | 1.65 | 6.80 | 280 | |||
4 | ✓ | ✓ | ✓ | 87.64 | 1.63 | 6.91 | 268 | |||
5 | ✓ | ✓ | ✓ | ✓ | 87.96 | 1.60 | 6.75 | 258 | ||
6 | ✓ | ✓ | ✓ | ✓ | 88.35 | 1.97 | 7.60 | 243 | ||
7 | ✓ | ✓ | ✓ | ✓ | 88.43 | 1.68 | 6.92 | 247 |
Model | mIou | mPA/% | Paras/M | FLOPs/G | FPS |
---|---|---|---|---|---|
U-Net [20] | 90.30 | 94.73 | 24.89 | 225.85 | 61 |
SegNet [21] | 88.79 | 94.17 | 29.46 | 321.65 | 88 |
DeepLab V3+ [22] | 88.85 | 94.35 | 5.81 | 52.87 | 176 |
BiSeNet [23] | 87.86 | 94.23 | 23.08 | 40.7 | 197 |
LETNet [24] | 88.94 | 93.29 | 0.95 | 13.6 | 156 |
ELANet [25] | 87.39 | 93.11 | 0.67 | 9.8 | 100 |
ENet [26] | 86.74 | 93.46 | 0.37 | 0.22 | 105 |
Improved LinkNet | 88.43 | 94.12 | 1.67 | 6.92 | 247 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Lu, X.; Zhu, S.; Lu, B. Pointer Meter Reading Method Based on YOLOv8 and Improved LinkNet. Sensors 2024, 24, 5288. https://doi.org/10.3390/s24165288
Lu X, Zhu S, Lu B. Pointer Meter Reading Method Based on YOLOv8 and Improved LinkNet. Sensors. 2024; 24(16):5288. https://doi.org/10.3390/s24165288
Chicago/Turabian StyleLu, Xiaohu, Shisong Zhu, and Bibo Lu. 2024. "Pointer Meter Reading Method Based on YOLOv8 and Improved LinkNet" Sensors 24, no. 16: 5288. https://doi.org/10.3390/s24165288