A License Plate Recognition System with Robustness against Adverse Environmental Conditions Using Hopfield’s Neural Network
Round 1
Reviewer 1 Report
In the paper was proposed 5 steps algorithm for license plate recognition. The article has the following main shortcomings:
1) Background analysis practically is absent. As a result, the proposed ideas are compared with outdated methods.
2) Traditionally, in such studies, a recognition using proposed algorithm is performed on a large dataset and recognition metrics are calculated. Here we see only single attempts to apply algorithm.
3) As a consequence of point 2, a comparison with the state of art algorithms in the field is not provided.
4) Proposed ways of processing seem as obsolete because nowadays best results are achieved using deep neural networks.
Author Response
In the paper was proposed 5 steps algorithm for license plate recognition. The article has the following main shortcomings:
Comment 1.1: Background analysis practically is absent. As a result, the proposed ideas are compared with outdated methods.
Response: The text has been modified to better present the background of paper based on the suggestion of Reviewer 1. Introduction section of the paper has fully been revised for background analysis, and more up to date papers [1, 8, 13-21] added based on the suggestion of Reviewer 3.
- Tsakanikas, V.; Dagiuklas, T. Video surveillance systems-current status and future trends. Electr. Eng. 2018, 70, 736-753.
- Aggarwal, A.; Rani, A.; Kumar, M. A robust method to authenticate car license plates using segmentation and ROI based approach. Smart Sustain. Built Environ. 2020, 9, 737-747.
- Kamilaris, A.; Prenafeta-Boldú, F. A review of the use of convolutional neural networks in agriculture. J. Agric. Sci. 2018, 156, 312-322.
- Hopfield, J.J. Neural networks and physical systems with emergent collective computational abilities. Natl. Acad. Sci. 1982, 79, 2554–2558.
- Lin, C.-J.; Chuang, C.-C.; Lin, H.-Y. Edge-AI-based real-time automated license plate recognition System. Appl. Sci. 2022, 12, 1445.
- Yousaf, U.; Khan, A.; Ali, H.; Khan, F.G.; Rehman, Z.U.; Shah, S.; Ali, F.; Pack, S.; Ali, S. A deep learning based approach for localization and recognition of Pakistani vehicle license plates. Sensors 2021, 21, 7696.
- Park, S.-H.; Yu, S.-B.; Kim, J.-A.; Yoon, H. An All-in-one vehicle type and license plate recognition system using YOLOv4. Sensors 2022, 22, 921.
- Wang, H.; Li, Y.; Dang, L.-M.; Moon, H. Robust korean license plate recognition based on deep neural networks. Sensors 2021, 21, 4140.
- Villanueva, A.; Daunys, G.; Hansen, D. W.; Böhme, M.; Cabeza, R.; Meyer, A.; Barth, E. A geometric approach to remote eye tracking. Univers. Access Inf. Soc. 2009, 8, 241-257.
- Daunys, G.; Ramanauskas, N. The accuracy of eye tracking using image processing. NordiCHI '04: Proceedings of the third Nordic conference on human-computer interaction, 2004, 377–380.
- Ramanauskas, N.; Daunys, G.; Dervinis, D. Investigation of calibration techniques in video based eye tracking system In: Miesenberger, K., Klaus, J., Zagler, W., Karshmer, A. (eds) Computers Helping People with Special Needs. ICCHP 2008. Lecture Notes in Computer Science, vol 5105. Springer, Berlin, Heidelberg, 2008.
Comment 1.2: Traditionally, in such studies, a recognition using proposed algorithm is performed on a large dataset and recognition metrics are calculated. Here we see only single attempts to apply algorithm.
Response: “The first step in recognizing a car license plate is to distinguish the car from other ob-jects in the image. For this, the methods presented in previous works can be used [8-12]. In similar works, the use of convolutional neural network (CNN) replaces parts of the pro-posed method in this paper. Despite the ease of use of this new neural network, there are major disadvantages in CNNs. The main disadvantage is that CNNs take much longer time to train. Another important disadvantage is the need for larger training datasets (i.e. hundreds or thousands of images), and their proper annotation, which is a delicate pro-cedure that must be performed by domain experts. Other disadvantages include problems that might occur when using pretrained models on similar and smaller datasets (i.e. a few hundreds of images or less), optimization issues due to model complexities, as well as hardware restrictions [13]. However, the proposed algorithm assumes there are only cars on highways or the absence of objects corresponding to license plates in non-car elements (such as humans, etc.). This contribution can overcome the burdens that are present in previous works.”
The above-mentioned response has also been added to the main text of the revised paper (see pp. 2).
Comment 1.3: As a consequence of point 2, a comparison with the state of art algorithms in the field is not provided.
Response: The method had been compared with CNN and SOTA methods, and the benefits of utilized method have been mentioned and highlighted.
the following added to the revised version of the manuscript (see pp. 10):
“The agile training capability of the Hopfield neural network has made it appropriate to be applied to plates with different standards, while for other neural networks such as convolutional neural networks, a huge set of training data must be collected for each standard of the plates. Moreover, the accuracy of the proposed algorithm is higher than a number of similar ones developed on SOTA [15,16]. On the other hand, the time spent to recognize the characters of each license plate is almost equal to the time spent to recognize only one character in methods based on convolutional neural networks [15-18].”
Comment 1.4: Proposed ways of processing seem as obsolete because nowadays best results are achieved using deep neural networks.
Response: The method had been compared with convolutional neural network methods, and the benefits of utilized method have been mentioned and highlighted.
the following added to the revised version of the manuscript (see pp. 2):
“In similar works, the use of convolutional neural network (CNN) replaces parts of the proposed method in this paper. Despite the ease of use of this new neural network, there are major disadvantages in CNNs. The main disadvantage is that CNNs take much longer time to train. Another important disadvantage is the need for larger training datasets (i.e. hundreds or thousands of images), and their proper annotation, which is a delicate procedure that must be performed by domain experts. Other disadvantages include problems that might occur when using pretrained models on similar and smaller datasets (i.e. a few hundreds of images or less), optimization issues due to model complexities, as well as hardware restrictions [13].”
Reviewer 2 Report
L27: Citation for: "Nowadays, the use of surveillance-based security systems has become increasingly important in various applications, such as home security and traffic monitoring"
L30: citation.
L36: citation.
L36: define CPR.
L40: citation for "recent advances...."
L48: not really clear what all those citations relate to.
L57: citation.
L60: citation for Hopfield.
L68/69: Again, not clear to what the four citations refer? These should be distributed (see above comments).
L72: typo: should be "mage" should be "image"?
L78: "large number" ......how many?
L155: Not really clear what all the citations refer to here.
L167/168: All mentioned need citations.
L169: needs citation.
L175: Multiple citations really necessary, where one might do for this ref.
L186: Should be equation 11?
L203: "matte" should be "matter".
This is an interesting piece of work that makes use of the Hopfield network for car registration plate identification. There is very good use of images and figures (diagrams) to aid the discussion.
Perhaps, some of the Introduction material (augmented with further detailed discussion) could be placed in a new Literature Review (Background) section?
In addition to the equations, further, more detailed text-based discussion of the operation of the Hopfield network would enhance the paper (e.g. in Section 2.5).
As well as the example described, details (e.g. tabular or graphical presentation) of many more tests (dataset?) carried out would be useful, together with the success rate, etc.
Comparing the above data to the results from other works in the literature would further strengthen the claims made.
Overall, an interesting and, generally, well-presented paper, which could benefit from some further work.
Author Response
Comment 2.1: L27: Citation for: "Nowadays, the use of surveillance-based security systems has become increasingly important in various applications, such as home security and traffic monitoring"
Response:
- Tsakanikas, V.; Dagiuklas, T. Video surveillance systems-current status and future trends. Electr. Eng. 2018, 70, 736-753.
Comment 2.2: L30: citation.
Response:
- Tsakanikas, V.; Dagiuklas, T. Video surveillance systems-current status and future trends. Electr. Eng. 2018, 70, 736-753.
Comment 2.3: L36: define CPR.
Response: Licence Plate Recognition (LPR)
Comment 2.4: L40: citation for "recent advances...."
Response: Corrected
Comment 2.5: L48: not really clear what all those citations relate to.
Response: Corrected
Comment 2.6: L57: citation.
Response: Corrected
Comment 2.7: L60: citation for Hopfield.
Response:
- Hopfield, J.J. Neural networks and physical systems with emergent collective computational abilities. Natl. Acad. Sci. 1982, 79, 2554–2558.
Comment 2.8: L68/69: Again, not clear to what the four citations refer? These should be distributed (see above comments).
Response: Corrected.
References 8-12 are related to the pervious works in the field.
Comment 2.9: L72: typo: should be "mage" should be "image"?
Response: Corrected.
Comment 2.10: L78: "large number" ......how many?
Response: Details have been given in the last part of methodology section.
Comment 2.11: L155: Not really clear what all the citations refer to here.
Response: References [28-32] have used the methods for determining the segments inside the plate
Comment 2.12: L167/168: All mentioned need citations.
Response: Corrected.
- Aksoy, Y.; Aydin, T.O.; Pollefeys, M. Designing effective inter-pixel information flow for natural image matting. Conference on Computer Vision and Pattern Recognition, 2017.
- Murty, M.N.; Devi, V.S. Pattern Recognition. Undergraduate Topics in Computer Science, vol 0. Springer, London, 2011.
Comment 2.13: L169: needs citation.
Response: Corrected.
Comment 2.14: L175: Multiple citations really necessary, where one might do for this ref.
Response: Corrected.
References [34-38] have used similar methods for different problems.
Comment 2.15: L186: Should be equation 11?
Response: Corrected.
Comment 2.16: L203: "matte" should be "matter".
Response : Corrected.
This is an interesting piece of work that makes use of the Hopfield network for car registration plate identification. There is very good use of images and figures (diagrams) to aid the discussion.
Comment 2.17: Perhaps, some of the Introduction material (augmented with further detailed discussion) could be placed in a new Literature Review (Background) section?
Response: The text has been modified to better present the background of paper based on the suggestion of Reviewer 1. Introduction section of the paper has fully been revised for background analysis, and more up to date papers [1, 8, 13-21] added based on the suggestion of Reviewer 2.
- Tsakanikas, V.; Dagiuklas, T. Video surveillance systems-current status and future trends. Electr. Eng. 2018, 70, 736-753.
- Aggarwal, A.; Rani, A.; Kumar, M. A robust method to authenticate car license plates using segmentation and ROI based approach. Smart Sustain. Built Environ. 2020, 9, 737-747.
- Kamilaris, A.; Prenafeta-Boldú, F. A review of the use of convolutional neural networks in agriculture. J. Agric. Sci. 2018, 156, 312-322.
- Hopfield, J.J. Neural networks and physical systems with emergent collective computational abilities. Natl. Acad. Sci. 1982, 79, 2554–2558.
- Lin, C.-J.; Chuang, C.-C.; Lin, H.-Y. Edge-AI-based real-time automated license plate recognition System. Appl. Sci. 2022, 12, 1445.
- Yousaf, U.; Khan, A.; Ali, H.; Khan, F.G.; Rehman, Z.U.; Shah, S.; Ali, F.; Pack, S.; Ali, S. A deep learning based approach for localization and recognition of Pakistani vehicle license plates. Sensors 2021, 21, 7696.
- Park, S.-H.; Yu, S.-B.; Kim, J.-A.; Yoon, H. An All-in-one vehicle type and license plate recognition system using YOLOv4. Sensors 2022, 22, 921.
- Wang, H.; Li, Y.; Dang, L.-M.; Moon, H. Robust korean license plate recognition based on deep neural networks. Sensors 2021, 21, 4140.
- Villanueva, A.; Daunys, G.; Hansen, D. W.; Böhme, M.; Cabeza, R.; Meyer, A.; Barth, E. A geometric approach to remote eye tracking. Univers. Access Inf. Soc. 2009, 8, 241-257.
- Daunys, G.; Ramanauskas, N. The accuracy of eye tracking using image processing. NordiCHI '04: Proceedings of the third Nordic conference on human-computer interaction, 2004, 377–380.
- Ramanauskas, N.; Daunys, G.; Dervinis, D. Investigation of calibration techniques in video based eye tracking system In: Miesenberger, K., Klaus, J., Zagler, W., Karshmer, A. (eds) Computers Helping People with Special Needs. ICCHP 2008. Lecture Notes in Computer Science, vol 5105. Springer, Berlin, Heidelberg, 2008.
Comment 2.18: In addition to the equations, further, more detailed text-based discussion of the operation of the Hopfield network would enhance the paper (e.g. in Section 2.5).
Response: The text has been modified to better highlight the more detailed text-based discussion of the operation of the Hopfield network based on the suggestion of Reviewer 2. The following has also been added to the manuascript (see pp 10):
“The segments which are in black and white format, are resized to a standard size. The matrix of each segment, which is two-dimensional, is transformed into a one-column vector. The black pixels marked with 0 in this matrix are changed to -1 and then applied to the Hopfield neural network. On the other hand, in this neural network, the main characters in standard size and 0 values corresponding to black pixels, which are replaced by -1, are defined as balanced points. The Hopfield neural network moves the input matrix to the nearest balanced state. In other words, the closest standard character similar to the target segment is recognized.”
Comment 2.19: As well as the example described, details (e.g. tabular or graphical presentation) of many more tests (dataset?) carried out would be useful, together with the success rate, etc.
Response: Other tests has also been performed and added to the revised version of the paper as follows (see pp 10):
“To determine the accuracy of Hopfield's neural network in determining numbers and letters, according to Figure 12, a set of different car license plates has been considered. The graphics on these plates can play the role of noise. Hopfield's neural network has classified the 253 characters on the license plates of this collection (after image processing and segmentation). Among the 253 test characters, only 6 characters were recognized wrongly. Therefore, the accuracy is 97.6%. Using a computer with Core(TM) i7-2640 CPU and 8GB RAM, the time spent to determine the characters of each license plate is about 0.08 seconds.
In addition to the 253 main characters (numbers or letters), there are also 26 special characters (such as a dash or a combination of numbers and letters). According to the different standards in these plates, all special characters were considered as a unit pattern. In addition to correctly recognizing the main characters, Hopfield's neural network must also classify special characters in this unit pattern. The correct classification rate considering these special characters is reported as 97.1%.”
Figure 12. A set of selected license plates from different standards to determine CCR of the proposed algorithm.
Comment 2.20: Comparing the above data to the results from other works in the literature would further strengthen the claims made.
Response: The agile training capability of the Hopfield neural network has made it appropriate to be applied to plates with different standards, while for other neural networks such as convolutional neural networks, a huge set of training data must be collected for each standard of the plates. Moreover, the accuracy of the proposed algorithm is higher than a number of similar ones developed on SOTA [15,16]. On the other hand, the time spent to recognize the characters of each license plate is almost equal to the time spent to recognize only one character in methods based on convolutional neural networks [15-18].
Overall, an interesting and, generally, well-presented paper, which could benefit from some further work.
Author Response File: 
Author Response.pdf
Reviewer 3 Report
1. The abstract should be more concise, write your outcome and novelty of the work
2. The contribution should be highlighted in the last paragraph of introduction
3. There are some typos need to fix, revise the article grammatically
4. Add more recent works such as "Secure video communication using firefly optimization and visual cryptography", "Identification of copy-move and splicing based forgeries using advanced SURF and revised template matching", "A robust method to authenticate car license plates using segmentation and ROI based approach"
5. The method section need to elaborate with more mathematical foundations
6. Results need some improvements, compare the work with SOTA methods
Author Response
Comment 3.1: The abstract should be more concise, write your outcome and novelty of the work
Response: The abstract has been modified to better highlight the outcomes and novelty of the current work based on the suggestion of Reviewer 3.
“License plates typically have unique color, size, and shape characteristics in each country. This paper proposes a method for character extraction and pattern matching in license plate recognition systems. The method is based on a combination of morphological operations and edge detection techniques, along with the bounding box method for identifying and revealing license plate characters while removing unwanted artifacts such as dust and fog. Previous works on license plate recognition have utilized non-intelligent pattern matching techniques. However, to address issues such as incomplete readability of license plate numbers due to external factors like mud and dirt, this paper employs Hopfield neural network for pattern matching. The proposed technique can be applied in a variety of settings, including traffic monitoring, parking management, and law en-forcement, among others. The applied algorithm, unlike SOTA-based methods, does not need a huge set of training data and is implemented only by applying standard templates. Other ad-vantages of the proposed algorithm are high speed training process, ability to adapt with different standards, high speed response, and higher accuracy compared to similar algorithms.”
Comment 3.2: The contribution should be highlighted in the last paragraph of introduction
Response: The text has been modified to better highlight the contribution and applicability of the current work based on the suggestion of Reviewer 3. The following has also been added to the main text in introduction section (see pp. 2):
“The first step in recognizing a car license plate is to distinguish the car from other ob-jects in the image. For this, the methods presented in previous works can be used [8-12]. In similar works, the use of convolutional neural network (CNN) replaces parts of the pro-posed method in this paper. Despite the ease of use of this new neural network, there are major disadvantages in CNNs. The main disadvantage is that CNNs take much longer time to train. Another important disadvantage is the need for larger training datasets (i.e. hundreds or thousands of images), and their proper annotation, which is a delicate pro-cedure that must be performed by domain experts. Other disadvantages include problems that might occur when using pretrained models on similar and smaller datasets (i.e. a few hundreds of images or less), optimization issues due to model complexities, as well as hardware restrictions [13]. However, the proposed algorithm assumes there are only cars on highways or the absence of objects corresponding to license plates in non-car elements (such as humans, etc.). This contribution can overcome the burdens that are present in previous works.”
Comment 3.3: There are some typos need to fix, revise the article grammatically
Response: The revised version of the paper has been thoroughly proofread, and modified gramatically.
Comment 3.4: Add more recent works such as "Secure video communication using firefly optimization and visual cryptography", "Identification of copy-move and splicing based forgeries using advanced SURF and revised template matching", "A robust method to authenticate car license plates using segmentation and ROI based approach"
Response : The text has been modified and most recent related research works in the field have been cited based on the comment of Reviewer 3.
- Aggarwal, A.; Rani, A.; Kumar, M. A robust method to authenticate car license plates using segmentation and ROI based approach. Smart Sustain. Built Environ. 2020, 9, 737-747.
Comment 3.5: The method section need to elaborate with more mathematical foundations
Response: The text has been modified to better present the foundations of the current work based on the comment of Reviewer 3. The following has also been added to the main text in Methodology and Simulation Results section (see pp 10):
“To determine the accuracy of Hopfield's neural network in determining numbers and letters, according to Figure 12, a set of different car license plates has been considered. The graphics on these plates can play the role of noise. Hopfield's neural network has classi-fied the 253 characters on the license plates of this collection (after image processing and segmentation). Among the 253 test characters, only 6 characters were recognized wrongly, showing the accuracy of 97.6%. Using a computer with Core(TM) i7-2640 CPU and 8GB RAM, the time spent to determine the characters of each license plate is about 0.08 sec-onds.
In addition to the 253 main characters (numbers or letters), there are also 26 special characters (such as a dash or a combination of numbers and letters). According to the dif-ferent standards in these plates, all special characters were considered as a unit pattern. In addition to correctly recognizing the main characters, Hopfield's neural network can also classify special characters in this unit pattern. The updated classification rate considering the special characters is 97.1%.”
Figure 12. A set of selected license plates from different standards to determine CCR of the proposed algorithm.
Comment 3.6: Results need some improvements, compare the work with SOTA methods.
Response: The method had been compared with SOTA methods, and the following paragraph is added to the revised version of the manuscript (see pp. 10):
“The agile training capability of the Hopfield neural network has made it appropriate to be applied to plates with different standards, while for other neural networks such as convolutional neural networks, a huge set of training data must be collected for each standard of the plates. Moreover, the accuracy of the proposed algorithm is higher than a number of similar ones developed on SOTA [15,16]. On the other hand, the time spent to recognize the characters of each license plate is almost equal to the time spent to recognize only one character in methods based on convolutional neural networks [15-18].”
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
Scientific soundness of the paper after revision was increased. References about gaze tracking weakly related to paper and could be removed.
Author Response
Thank you for your feedback. We have removed 3 references about gaze tracking per reviewer's comments.
Reviewer 3 Report
The authors revised the article as per my previous suggestions. I have no further observations.
Author Response
Thank you for your constructive feedback.
