Enhanced YOLOv8-Based System for Automatic Number Plate Recognition

Abstract

This paper presents an advanced automatic number plate recognition (ANPR) system designed specifically for Qatar’s diverse license plate landscape and challenging environmental conditions. Leveraging the YOLOv8 deep learning model, particularly the YOLOv8s variant, we achieve state-of-the-art accuracy in both license plate detection and number recognition. Our innovative approach includes a comprehensive dataset enhancement technique that simulates adverse conditions, significantly improving the model’s robustness in real-world scenarios. We integrate edge computing using a Raspberry Pi with server-side processing, demonstrating an efficient solution for real-time ANPR applications. The system maintains greater than 93% overall performance across various environmental conditions, including night-time and rainy scenarios. We also explore the impact of various pre-processing techniques, including edge detection, k-mean thresholding, DBSCAN, and Gaussian mixture models, on the ANPR system’s performance. Our findings indicate that modern deep learning models like YOLOv8 are sufficiently robust to handle raw input images and do not significantly benefit from additional pre-processing. With its high accuracy and real-time processing capability, the proposed system represents a significant advancement in ANPR technology and is particularly suited for Qatar’s unique traffic management needs and smart city initiatives.

1. Introduction

Automatic number plate recognition (ANPR) systems have become increasingly important for effective traffic management, law enforcement, and security applications [1,2]. These systems automate the process of identifying vehicle license plates (LPs), which is crucial for monitoring and managing vehicle traffic, especially in rapidly growing urban areas [3,4]. In Qatar, implementing ANPR technology is particularly significant given the country’s diverse traffic scenario [5] characterized by various vehicle types and LP designs.

The development and deployment of ANPR systems in Qatar align with the country’s vision to harness advanced technology to improve its urban infrastructure [6]. ANPR technology plays a vital role in enhancing road safety, which has become a major concern in Qatar due to the rise in traffic-related incidents [7]. By enabling real-time monitoring and enforcement of traffic rules, ANPR systems contribute to reducing traffic violations that can lead to accidents [8,9,10].

However, the implementation of ANPR technology in Qatar faces several challenges [1]. The country’s unique environmental conditions, such as high temperatures and sandstorms, require robust and adaptable ANPR solutions. Moreover, the system must be capable of accurately recognizing different types of LPs, especially in densely populated areas. This necessitates a specialized approach to address these challenges effectively.

Current ANPR methods face several limitations in real-world applications, particularly in challenging environments like Qatar. These include reduced accuracy in adverse weather conditions, difficulties recognizing diverse LP designs, and computational inefficiencies that hinder real-time processing. Traditional methods often struggle with high variability in LP appearance due to factors such as lighting conditions, LP orientations, and physical damage. Moreover, many existing systems are not optimized for the specific characteristics of Qatari LPs, leading to reduced performance in local contexts. Our proposed method aims to address these limitations by leveraging advanced deep learning (DL) techniques, specifically tailored dataset enhancement, and a distributed computing architecture to achieve robust performance across diverse conditions while maintaining real-time processing capabilities.

Developing an advanced ANPR system tailored for Qatar’s unique environment is not just a technological advancement but a crucial step towards creating a sustainable and resilient urban infrastructure. By enabling more efficient traffic management and law enforcement, such a system can significantly reduce traffic congestion, lower emissions, and improve overall urban mobility. This aligns with Qatar’s vision for sustainable urban development, as outlined in the Qatar National Vision 2030 [11,12]. Furthermore, an effective ANPR system can inform data-driven urban planning decisions, supporting the creation of smarter, more adaptable city infrastructures that can evolve with changing transportation needs and environmental challenges.

This paper presents the development of an autonomous and independent ANPR system tailored explicitly for Qatari LPs. The research builds upon the groundwork laid by previous studies conducted by Al-Hasan et al. [1], which made significant contributions to this field. The primary objective of this study is to enhance the previous work and create an ANPR system that can efficiently handle the specific requirements of Qatar’s traffic environment. This work is focused more on localizing the ANPR process instead of relying on third-party applications/APIs for ANPR as done in [1].

The key contributions of this paper are as follows:

• Developing a robust ANPR system adapted to recognize Qatari LPs accurately, even in challenging environmental conditions;
• Integrating advanced machine learning (ML) techniques, such as the YOLOv8 model, to improve the system’s performance and adaptability;
• Creating enhanced datasets that simulate real-life scenarios, including low-quality LP images, to increase the system’s robustness;
• Exploring innovative pre-processing techniques, such as edge detection, k-mean thresholding, and Bayesian optimization, to refine the specific pipeline for Qatari ANPR;
• Developing a comprehensive, ready-to-deploy ANPR system with API connectivity for effective integration into the existing infrastructure.

The paper is structured as follows: Section 2 provides a background and literature review of ANPR systems, highlighting the current state-of-the-art and identifying gaps in existing research. Section 3 describes the methodology employed in this study and covers dataset preparation and pre-processing techniques. Then, Section 4 discusses the integration. The results of the experiments are presented in Section 5 along with a discussion of the findings and their implications, and we outline future research directions. Finally, Section 6 concludes the paper, summarizing the key contributions.

2. Background and Literature Review

ANPR systems have been the subject of extensive research due to their wide range of application in traffic management, law enforcement, and security [1,13]. The conventional ANPR methodology typically involves a sequence of steps, including image capture, pre-processing, LP detection, character segmentation [14], and character recognition [15,16]. These steps are crucial for accurate vehicle identification and require robust image processing techniques.

Over the years, various approaches have been proposed to enhance the performance of ANPR systems. Agrawal et al. [17] employed ML and data visualization techniques to improve the cognitive aspect of LP recognition. Ahmad et al. [18] conducted a comparative study of different ANPR techniques, highlighting the advantages and limitations of each approach. Babu and Raghunadh [19] utilized a bounding box method for vehicle LP detection and recognition and demonstrated its effectiveness at handling various plate formats.

The introduction of DL techniques has revolutionized the field of ANPR. Models like YOLO (you only look once) have gained significant attention due to their ability to perform real-time object detection accurately. Laroca et al. [20] proposed an efficient and layout-independent ANPR system based on the YOLO detector and showcased its robustness at detecting LPs under diverse conditions. Ding et al. [21] introduced an end-to-end contrastive LP detector using YOLO, further enhancing the model’s performance.

The evolution of the YOLO object detection framework has been marked by significant advancements, particularly with the introduction of YOLOv8 in 2023, which was released by Jocher et al. (Ultralytics) [22]. This latest iteration has garnered attention due to its enhanced capabilities in various applications, including ANPR. YOLOv8 is designed to improve detection speed and accuracy, which are critical factors in real-time applications such as traffic monitoring and enforcement systems [23,24,25].

YOLOv8 builds upon the foundational principles established by its predecessors, emphasizing real-time performance and high classification accuracy while maintaining computational efficiency. The architecture of YOLOv8 incorporates several innovations, such as improved backbone networks and attention mechanisms that enhance its ability to focus on relevant features in complex images [26,27]. These enhancements are particularly beneficial for applications like ANPR, where the detection of small and often occluded objects, such as LPs, is required [23,24,27].

In the context of improved small object detection (ISOD), YOLOv8’s is able to significantly impact the efficiency and effectiveness of ANPR systems; these improvements began with the introduction of improved small object detection in YOLOv7 by Ma et al. [28]. Integrating YOLOv8 into traffic monitoring systems allows for real-time detection and tracking of vehicles as well as the recognition of LPs, which is essential for law enforcement and traffic management [23,24,29]. The model’s ability to process video feeds from traffic cameras with high accuracy and speed enables authorities to monitor traffic violations more effectively, thus enhancing public safety and compliance with traffic regulations [23,25,29].

Moreover, YOLOv8’s architecture allows for the utilization of diverse datasets, which can be crucial for training models that are robust to various environmental conditions and vehicle types [23,24,30]. This adaptability is vital for ANPR systems operating in different lighting conditions and at different angles and distances. The ability to train on a wide range of data ensures that the model can generalize well, leading to improved performance in real-world scenarios [23,24,30].

The advancements in YOLOv8 also include ISOD, which is particularly relevant for ANPR applications, where an LP can be relatively small compared to the vehicle’s overall size [26,31]. Introducing attention mechanisms within the YOLOv8 framework enhances its capability to detect these small objects more accurately, thereby reducing the rate of false negatives in LP detection [26,27]. This is crucial for applications where accurate identification of vehicles is necessary for enforcement actions, such as issuing fines for traffic violations. Our work leverages such advancements, particularly by optimizing YOLOv8 for detecting the small and often obscured characters on LP under various environmental conditions.

Several studies have focused on the application of ANPR in specific contexts. Al-Hasan et al. [1] developed a smart speed camera system using ANPR for residential compounds and institutions in Qatar. Mustafa and Karabatak [32] proposed a DL model for ANPR at campus gates. Sferle and Moisi [33] implemented an ANPR system for a smart service auto and highlighted its potential in intelligent transportation systems.

Recent advancements in ANPR have seen the application of more sophisticated YOLO variants. YOLOv7, introduced by Wang et al. [34], demonstrated significant improvements in both speed and accuracy for object detection tasks, including LP recognition. Sankhe et al. [35] demonstrated a novel approach to ANPR by integrating YOLOv7 with optical character recognition (OCR) techniques, enhancing detection accuracy under various environmental conditions. Moreover, Hingorani et al. [36] utilized YOLOv7 and OCR technologies as well to streamline toll collection processes in India, aiming to enhance efficiency and reduce congestion at toll booths. Building upon this, YOLOv8 further enhanced performance with architectural improvements and training strategies. For instance, Angelika Mulia et al. [37] utilized YOLOv8 for LP detection, achieving notable accuracy improvements. Similarly, Chopade et al. [38] integrated YOLOv8 with ResNet-50 for high-accuracy LP recognition under varying illumination conditions. These studies highlight the potential of the latest YOLO variants for addressing the challenges of ANPR in diverse environments.

Despite the advancements in ANPR technology, challenges persist in terms of adaptability to diverse environments, robustness to varying image quality, and the need for comprehensive datasets. Tang et al. [39] systematically reviewed ANPR in smart cities, emphasizing technological advancements and application cases. The study identified gaps in the current literature, such as the lack of comparative analyses between various YOLO versions and the requirement for adaptable models.

Moreover, the gap analysis in the current literature highlights several key challenges:

• Challenges with studying newer versions of YOLO, such as YOLOv8, due to their recent emergence and the limited research available;
• Lack of comparative analyses of various YOLO versions, which are crucial for understanding the strengths and limitations of each version and selecting the most suitable model for specific ANPR requirements;
• Need for enhanced datasets adaptable to real-life situations and that can continuously evolve to improve model performance;
• Requirement for adaptable models that can handle diverse environments and challenges, providing a framework for continuous improvement.

To address these challenges, researchers have explored innovative approaches. Cheng and Chen [40] utilized generative adversarial network (GAN) technology for repairing dynamically blurred LPs. Liu et al. [41] studied improvements to the YOLOv3 algorithm, enhancing its performance in LP recognition. Ma et al. [42] proposed a DL-based framework for end-to-end identification of IMO numbers on ships, demonstrating the versatility of ANPR in maritime applications.

An overview comparison of selected studies in the field of ANPR can be observed in

Table 1. Summary of selected articles and studies that were reviewed by the authors.

Ref.	Study Description	Methodology	Key Findings
[1]	Smart speed camera system using ANPR for residential compounds and institutions in Qatar	ANPR integration with speed cameras	Highlighted the potential of ANPR in enhancing traffic safety in residential areas
[15]	Automatic recognition system for recording incoming and outgoing vehicles	Conventional ANPR methodology	Demonstrated the effectiveness of ANPR in university campus settings
[20]	Efficient and layout-independent ANPR system based on YOLO detector	DL (YOLO)	Showcased the robustness of YOLO in detecting LPs under diverse conditions
[35]	Combines YOLOv7 and OCR to improve ANPR, addressing challenges in real-world scenarios	DL (YOLOv7, Pytesseract)	Improved detection accuracy and robustness against environmental challenges through data augmentation
[36]	An ANPR system to improve toll collection efficiency in India	DL (YOLOv7)	Achieved high precision in LP detection and reduced processing time
[37]	Evaluates the performance of YOLOv8 and Faster R-CNN object detection models for LP recognition and the impact of super-resolution on OCR accuracy	DL (YOLOv8, Faster R-CNN)	Incorporated super-resolution into an ANPR system, reducing the character error rate to 51.7%
[38]	Proposes an ANPR system that integrates YOLOv8 for object detection and ResNet-50 for feature extraction, achieving high accuracy on Indian LPs	DL (YOLOv8, ResNet-50)	Succeeded in localizing single-line LPs and achieving high-accuracy character recognition under varying illumination conditions
[39]	Systematic review of ANPR in smart cities	Literature review	Identified gaps in current research and emphasized the need for adaptable models
[40]	Repairs dynamically blurred LPs using GAN technology	GANs	Demonstrated the potential of GANs in enhancing LP image quality
[43]	Comparison of traditional ANPR approaches with YOLOv3 for real-time Indian LP detection	DL (YOLOv3)	Traditional approaches focused on contouring, segmentation, and edge detection processes yielded lower accuracy, while YOLOv3 provided more accurate results for Indian LP detection in real-time
[44]	Develops an ANPR framework for Pakistani vehicle LPs using YOLO object detection and OCR	DL (LeNet-CNN, YOLOv4)	The proposed ANPR framework using YOLOv4 and OCR Tesseract demonstrated effective performance across diverse Pakistani LP formats
[45]	Compares the performance of YOLO and R-CNN algorithms for vehicle LP detection	DL (YOLOv4, R-CNN)	YOLOv4 had higher accuracy than R-CNN
[46]	Develops ANPR for Jordanian LPs and a recognition system using DL techniques, specifically the YOLOv3 model and transfer learning to enhance accuracy and efficiency	DL (YOLOv3, VGG16)	The proposed system effectively detects and recognizes Jordanian LPs and vehicle logos with high accuracy by utilizing DL models trained on extensive datasets
[47]	Presents an end-to-end ANPR system using YOLOv4 for vehicle and LP detection, enhancing accuracy and robustness in real-world settings	DL (YOLOv4)	Achieved significant improvements in detection accuracy across diverse datasets, showcasing its generalizability under varying conditions
[48]	Develops a smart parking system using a Raspberry Pi with RetinaNet for ANPR and Keras OCR for LP recognition.	DL (RetinaNet)	Achieved high accuracy and efficient processing and is suitable for various parking management applications
[49]	Indonesian LP recognition using CNNs to handle various noise conditions	DL (CNN with sliding window method)	Achieved high accuracy on normal data and low accuracy on noisy data; sliding window performance decreased under noisy conditions

. The literature review reveals the significant progress made in ANPR technology, particularly in integrating DL techniques. However, it also highlights the need for further research to address the challenges of adaptability, robustness, and comprehensive datasets, especially in the context of diverse environments like Qatar. This paper aims to contribute to this research gap by developing an advanced ANPR system tailored for Qatari LPs incorporating state-of-the-art ML techniques and innovative pre-processing approaches.

3. Methodology

In the methodology section, the research followed a structured approach to design and implement an ANPR system tailored for Qatari LPs, as described earlier. Our approach to developing an advanced ANPR system for Qatar involved three main phases: dataset creation and annotation, model selection and training, and dataset enhancement. Each phase was meticulously designed to address the unique challenges posed by Qatar’s diverse LP designs and environmental conditions.

Figure 1 illustrates the overall architecture of our proposed ANPR-based solution. This system integrates edge computing using a Raspberry Pi for image capture with server-side processing for ANPR tasks. The workflow includes image capture, data transmission, ANPR processing, and results communication, forming a comprehensive solution for real-time LP recognition in Qatar’s diverse environmental conditions. This is done to replace the third-party ANPR API implemented in [1] and to instead localize the ANPR process by utilizing the existing setup without any additional hardware.

3.1. Dataset Creation and Annotation

The foundation of our ANPR system lies in a robust and comprehensive dataset. We utilized Roboflow, a powerful dataset management and annotation platform, to prepare our training data.

Data collection focused on capturing the diversity of Qatari LPs. We collected a substantial dataset of 2432 images from the security gates and parking areas of Qatar University, as described in [1]. Figure 2 illustrates various categories of Qatari LPs present in the dataset. All illustrated LPs have the country’s name “QATAR" and its type printed in Arabic and English, with some of the types being in Arabic only. Additionally, Qatari LPs are exclusively numbered using Arabic numerals ‘0123456789’, and have a minimum of three digits to a maximum of six digits. These images covered a range of lighting conditions, from well-lit to low-light scenarios, and included various LP types, as shown in Figure 2. This diversity was essential for the model’s adaptability to real-world scenarios.

The annotation process was meticulous and time-intensive. We manually annotated each of the 2432 images using Roboflow’s user-friendly interface, creating bounding boxes around both the LPs and individual numbers. This level of detail was essential for training a model capable of detecting LPs and recognizing individual numbers. The process involved several key steps: first, all images were uploaded to the Roboflow platform. Then, we drew precise bounding boxes around the entire LP for each image. Subsequently, individual numbers within each LP were annotated with separate bounding boxes. Each annotated element was then assigned to its respective class, whether an LP or a specific number. Finally, a thorough quality control review was conducted to ensure annotation accuracy and consistency across the dataset.

To prepare the dataset for model training, we implemented a split of 70% for training, 20% for validation, and 10% for testing. This split was chosen to provide a substantial training set while retaining sufficient data for validation and testing. Notably, we deliberately avoided pre-processing and augmentation steps at this stage. This decision was made to establish the baseline performance of the YOLOv8 models on raw data, allowing us to assess the model’s inherent capability to handle diverse, unaltered images.

3.2. Model Selection and Training

The selection of an appropriate model was crucial for the success of our ANPR system. After carefully considering various options, we chose YOLOv8 as our primary model; this decision was supported by several factors. YOLOv8—developed by the developers of YOLOv5 [22,50]—builds upon the successes of its predecessors while offering significant speed and accuracy improvements, which are essential for the real-time processing requirements of an ANPR system [50]. Moreover, YOLOv8 provides a comprehensive framework that extends beyond conventional object detection models and covers the entire lifecycle of ML models—from data ingestion and model training to validation, deployment, and real-world application [22]. Additionally, the versatility of YOLOv8, which supports multiple modes such as ‘Train’, ‘Val’, ‘Predict’, ‘Export’, ‘Track’, and ‘Benchmark’, made it particularly adaptable to the various stages of our project development.

The structure in Figure 3 represents the YOLOv8 architecture [50]. The visualized key components include a backbone that utilizes CSPDarknet with C2f modules and an SPPF module. The neck features a modified FPN+PAN structure, and the architecture incorporates three detection heads designed for different scales.

For the training environment, we opted for Google Colab due to the accessibility to powerful computational resources, particularly GPUs. This choice was crucial for efficiently training complex models like YOLOv8. We utilized a Google Colab Pro subscription to access high-performance GPUs (A100 and V100), significantly accelerating our training process.

The training process for our model began with a thorough environment setup. We configured the Google Colab environment by installing the necessary packages and the YOLOv8 library (version 8.0.20 as of 27 January 2023). This step involved executing specific installation commands to ensure YOLOv8 and its dependencies were correctly installed. Once the environment was ready, we imported the annotated dataset from Roboflow by using Roboflow’s Python 3.8 package, which allowed us to download and unzip the dataset directly into the Colab environment.

With the environment and data in place, we initiated model training using a custom YOLOv8 command. A critical part of this process was hyperparameter tuning, whereby we experimented with various parameters such as the learning rate, batch size, and image size. This iterative process involved multiple training runs with different configurations to identify the optimal set of hyperparameters.

Throughout the training, we closely monitored key performance metrics, including the box loss, classification loss, and mean average precision (mAP), using YOLOv8’s built-in logging and visualization tools. Finally, we validated the model’s performance using the ‘Val’ mode to assess its accuracy and generalization capabilities on the validation set.

To determine the most suitable version of YOLOv8 for our ANPR system, we comprehensively compared different YOLOv8 variants (nano, small, medium, large, and x-large) for both LP detection and number recognition tasks. Our evaluation metrics included mAP50, mAP50-95, and inference speed. This comparative analysis involved training and evaluating each model variant using identical datasets and hyperparameters to ensure a fair comparison.

3.3. Exploration of Pre-Processing Techniques

We investigated various pre-processing techniques to further enhance the ANPR system’s performance. These methods were applied to the images before feeding them into the YOLOv8 model:

• Edge detection: We implemented and compared several edge detection algorithms, including Canny, high-pass, Laplacian, and Sobel filters. These techniques aim to enhance the clarity of features within the image, potentially improving LP and number recognition [51].
• K-mean thresholding: A k-mean clustering algorithm was applied to the image histograms for adaptive thresholding. We experimented with different numbers of clusters to find the optimal segmentation of LP features [51,52].
• DBSCAN (density-based spatial clustering of applications with noise): This clustering algorithm was used to analyze images in a three-dimensional space by considering both pixel location and intensity. This approach aimed to identify and enhance relevant features in the LP images [52,53,54].
• Gaussian mixture model (GMM): We applied a GMM to the grayscale histograms of images to achieve a more defined clustering of pixel intensities. Different numbers of components were tested to find the best segmentation of LP features [53,55].

These pre-processing techniques were systematically applied to our dataset, and their impact on the ANPR system’s performance was evaluated.

3.4. Enhanced Dataset Creation

To improve the robustness of our ANPR system, particularly its ability to handle challenging real-world scenarios, we developed a framework to simulate adverse conditions and create an enhanced dataset. This process involved several steps:

• Identification of challenging scenarios: We identified several challenging LP categories, listed in

  Table 2. Examples of LPs in several challenging scenarios.

  Challenging Scenario	Description	LP from Original Dataset	LP with Developed Function
  Flash and deteriorated (FD)	Simulating LPs affected by camera flash and physical deterioration
  Deteriorated with low contrast	Replicating plates with poor visibility due to wear and low contrast
  No painting	Mimicking plates for which the paint has worn off, leaving only embossed numbers
  Flash with bad focus	Simulating the effect of camera flash combined with poor focus
  Night-time	Dark environment; poor lighting		-
  Rainy	Weather condition affecting visibility		-
  Partial occlusion	LP blocked or obstructed by any object		-
  Glare	Direct sun glare on the camera or from reflections		-

  , based on common issues encountered in real-world ANPR applications.
• Development of simulation functions: We developed image processing functions using OpenCV to simulate various conditions. These functions transformed high-quality LP images to mimic desired effects. For example, the FD function combined binary thresholding with custom noise to simulate number erosion and flash effects, while the ‘No painting’ function used edge detection and contrast enhancement to replicate the look of unpainted, embossed numbers.
• Automated dataset enhancement: We developed a Python script to automate the enhancement of our dataset by applying simulated conditions. The script iterated through each image, applied various simulation functions to generate multiple variations, created new filenames for these modified images, and saved the modified images along with their corresponding label files.
• Label management: To maintain dataset integrity, we developed a method for managing labels with the modified images. This process involved copying the original label files for each modified image and adjusting the label coordinates if the simulation functions altered image dimensions or object positions.
• Dataset Expansion: Through this process, we significantly increased the size and diversity of our dataset. For each original image, multiple variations were created: each representing a different challenging scenario.
• Quality control: After the automated enhancement process, we manually reviewed a subset of the enhanced images to ensure the simulations accurately represented real-world challenges and that the labels remained accurate.

This comprehensive methodology, encompassing careful dataset creation, strategic model selection, and innovative dataset enhancement, formed the foundation of our advanced ANPR system tailored for Qatar’s unique LP landscape. By focusing on these key areas, we aimed to develop a robust system capable of handling the diverse and challenging conditions encountered in real-world ANPR applications in Qatar.

4. System Integration and Deployment

The final phase of our project involved integrating the trained ANPR model into a functional system and deploying it in a real-world setting. This section describes the process of creating a comprehensive ANPR script, establishing API connectivity, and integrating the system with a Raspberry Pi for practical deployment.

We developed a Python script that encapsulates the entire ANPR process from image input to LP number extraction. The script integrates several key functions:

• Car function: This function utilizes a YOLOv8 model trained on the COCO dataset to detect vehicles in the input image. It identifies various vehicle types, including cars, motorcycles, buses, and trucks.
• Plate function: This function employs our custom-trained YOLOv8 model to locate and isolate an LP within the detected vehicle area.
• Number function: This function uses another specialized YOLOv8 model to recognize and interpret the individual numbers on the isolated LP.
• ANPR function: This main function orchestrates the entire process, calling the above functions in sequence and handling any exceptions or fallback scenarios.

The script was designed with robustness in mind and incorporates error handling and a two-tiered approach to ensure maximum accuracy. The primary approach attempts to directly extract the LP numbers from the provided image. If this fails, a backup strategy is activated, which uses the ‘Car’ function first to identify and crop the vehicle area before re-attempting plate and number recognition. This process is illustrated in Figure 4 as a simple flowchart. This dual-strategy approach significantly enhances the system’s ability to handle diverse and challenging input scenarios.

To address the computational limitations of the Raspberry Pi and enable efficient real-time processing, we implemented an API-based solution. The first step involved setting up a high-performance server to host the ANPR processing capabilities on a high-performing workstation. This server was equipped with a Windows 11 64-bit operating system, 128 GB of RAM, an Intel i9-13900K CPU (featuring 24 cores, 32 threads, and a maximum turbo frequency of 5.7 GHz), and an NVIDIA 40-series GPU with 12 GB VRAM.

Next, we developed a Flask-based API on the server. Flask was chosen due to its simplicity and efficiency in creating web services. The API was designed to handle incoming requests efficiently and receives image data via HTTP POST requests, processes the images using our ANPR script, and returns the extracted LP numbers.

To ensure smooth integration with various client applications, we structured the API to return results in a JSON format. This approach not only addressed the computational constraints of the Raspberry Pi but also created a scalable solution capable of handling multiple requests simultaneously.

We utilized a Raspberry Pi 4B with 8 GB of RAM and a Broadcom BCM2711 processor for the edge device deployment; the Raspberry Pi featured a quad-core Cortex-A72 (ARM v8) 64-bit SoC clocked at 1.8 GHz. The integration process involved several key steps to ensure smooth operation and reliable communication with the server.

First, we configured the Raspberry Pi with Raspbian OS and installed the necessary libraries for image capture and network communication. We then integrated a Raspberry Pi camera module and configured it for efficient image capture. The next crucial step was the development of a Python client script for the Raspberry Pi. This script was designed to capture images at predefined intervals or when triggered by external sensors, to prepare and send these images to the server via HTTP POST requests, and to process the API responses. This script was simply merged with previous work [1].

To ensure reliable operation, we paid careful attention to the network configuration, optimizing the connection between the Raspberry Pi and the server for low latency and dependable data transfer. Additionally, we implemented robust error handling and a retry mechanism in the client script. These features were crucial for managing potential network issues or server unavailability, thereby ensuring continuity of service even in less-than-ideal conditions.

The integrated system operates as illustrated in Figure 5. The process begins with the Raspberry Pi capturing an image of a vehicle. This image is then sent to the server via an HTTP POST request. On receiving the image, the server processes it using the ANPR script. The extracted LP numbers are then returned to the Raspberry Pi, which subsequently displays and stores this information as required for the managerial mobile application and other processes [1].

This distributed architecture leverages Raspberry Pi’s compact form factor and low power consumption for deployment flexibility while offloading the computationally intensive ANPR tasks to a more powerful server. This approach ensures real-time processing capabilities even with the resource constraints of edge devices.

To ensure data security in the communication between the Raspberry Pi and the server, especially given the resource constraints of edge devices, implementing lightweight encryption methods is crucial. Recent work [56] on a security-driven performance analysis of lightweight cryptography on resource-constrained devices provides valuable insights for selecting appropriate encryption algorithms in such scenarios.

Moreover, Figure 6 presents a comprehensive flowchart of our ANPR system, encompassing the entire process from data preparation to real-time execution. This flowchart illustrates the four main phases of our system: pre-processing, model selection and training, system integration, and execution.

The flowchart is designed with the main phases arranged from left to right, providing a clear progression of the overall process. Within each phase, the specific steps are arranged from top to bottom, offering a detailed view of the workflow within each stage.

The pre-processing phase includes data collection, manual annotation using Roboflow, dataset splitting, and our novel dataset enhancement techniques for simulating adverse conditions. The model selection and training phase shows the process of selecting the YOLOv8s model, tuning hyperparameters, training it using Google Colab, and evaluating its performance.

The system integration phase depicts the setup of both the Raspberry Pi edge device and the high-performance server, as well as the development of the Flask-based API. Finally, the execution phase illustrates the real-time processing pipeline, from image capture on the Raspberry Pi to server-side ANPR processing and results communication.

This comprehensive visualization provides a clear overview of our system’s end-to-end workflow, highlighting the interconnections between different components and the logical flow of data and processes throughout the system.

This comprehensive integration and deployment strategy ensures that our ANPR system is not only accurate but also practically deployable in real-world scenarios and capable of handling the diverse and challenging conditions encountered in Qatar’s traffic management systems.

5. Results and Discussion

This section presents the outcomes of our ANPR system development and discusses the implications of our findings. We analyze the performance of different YOLOv8 models, evaluate the impact of our dataset enhancement techniques, and assess the overall system performance in real-world scenarios.

5.1. Overall Model Performance

Before delving into the specific comparisons between model variants, we present the general performance metrics of our ANPR system. These results provide a comprehensive overview of the model’s capabilities and effectiveness.

Figure 7 illustrates the training progression of our YOLOv8 model. The graphs show a consistent decrease in loss values (box loss, classification loss, and dfl loss) for both the training and validation sets, indicating effective learning and good generalization. The precision and recall metrics demonstrate steady improvement throughout the training process, with the model achieving high performance in both metrics by the end of training.

The notable fluctuations in the loss function convergence, as seen in Figure 7, can be attributed to several factors. Firstly, the diverse nature of our dataset, which includes challenging environmental conditions, likely contributes to these variations during training. As the model encounters different types of complex examples, it adjusts its parameters, leading to temporary increases in loss. Secondly, using data augmentation techniques introduces additional variability in the training samples, which can cause short-term fluctuations as the model adapts to these variations. Lastly, the learning rate schedule and batch size choices may contribute to these oscillations. Despite these fluctuations, the overall downward trend of the loss and the improvement in precision and recall metrics indicate that the model is effectively learning and generalizing from the data.

The F1 curve in Figure 8 demonstrates the model’s ability to balance precision and recall effectively. The high F1 scores across different confidence thresholds indicate robust performance in identifying LPs and numbers.

Figure 9 shows the precision of our model at various confidence thresholds. The high precision values, particularly at higher confidence levels, suggest that when our model makes a positive detection, it is highly likely to be correct.

The precision–recall curve in Figure 10 illustrates the trade-off between precision and recall at various threshold settings. The large area under this curve strongly indicates the model’s excellent performance, showcasing its ability to maintain high precision even as the recall increases.

Figure 11 demonstrates the model’s recall performance across different confidence thresholds. The high recall values indicate that our model effectively identifies a large proportion of the actual LPs and numbers in the images.

The confusion matrix in Figure 12 provides insight into the model’s classification accuracy for individual numbers on LPs. The strong diagonal pattern with high values indicates excellent classification performance across all number classes. Importantly, the lack of significant off-diagonal elements suggests minimal misclassification between different numbers, demonstrating the model’s ability to effectively distinguish between similar-looking digits (e.g., 8 and 0, 1 and 7, and 2 and 5).

These results collectively paint a picture of a highly effective ANPR system. The model demonstrates strong performance across various metrics, indicating its robustness in detecting and recognizing LPs and numbers under diverse conditions. The high auto-correlation observed in the confusion matrix, with minimal overlap between classes, further underscores the model’s accuracy and reliability in number recognition.

In the following subsections, we will explore how different variants of our model compare in terms of specific performance metrics and computational efficiency, building upon this foundation of overall strong performance.

5.2. Model Performance Comparison

We conducted a thorough comparison of different YOLOv8 variants for both LP detection and number recognition tasks.

5.2.1. LP Detection

Figure 13 presents the performance metrics for LP detection across different YOLOv8 models. The results demonstrate that all YOLOv8 variants achieve high accuracy in LP detection, with mAP50 values ranging from 0.992 to 0.997. The small (s) model stands out, with an optimal balance between accuracy (mAP50 of 0.994 and mAP50-95 of 0.777) and speed (inference time of 6.2 ms).

5.2.2. Number Recognition

For number recognition, we observed similar trends in model performance. Figure 14 summarizes the results.

Again, the small model provides an excellent trade-off between accuracy and speed, achieving a mAP50 of 0.986 and mAP50-95 of 0.715 with an inference time of 9.7 ms.

Based on these results, we selected the YOLOv8s model for LP detection and number recognition in our final ANPR system, as it offers the best balance between accuracy and computational efficiency.

5.3. Impact of Dataset Enhancement

Our dataset enhancement techniques, which involved simulating various challenging conditions, significantly improved the model’s performance on low-quality LPs.

Figure 15 illustrates examples of the simulated adverse conditions applied to LPs in our enhanced dataset with their order of operation: these conditions were applied left-to-right. Moreover,

Table 3. Performance comparison of models trained on original vs. enhanced datasets.

Model	mAP50	mAP75	mAP50-95
Original Dataset	0.87	0.71	0.65
Enhanced Dataset	0.92	0.79	0.73

compares the performance of models trained on the original dataset versus the enhanced dataset.

The improvement in mAP values through data augmentation can be attributed to several mechanisms. Firstly, by simulating adverse conditions such as poor lighting, weather effects, and image degradation, we expose the model to a broader range of scenarios that it might encounter in real-world applications. This increased diversity in the training data helps the model develop more robust features that are invariant to these challenging conditions. Secondly, augmentation effectively increases the size of our training dataset, providing more examples for the model to learn from and reducing the risk of overfitting specific image characteristics. Lastly, by including complex examples in the training data, we encourage the model to learn more discriminative features, improving its ability to accurately detect and recognize NPs in sub-optimal conditions. The model trained on the enhanced dataset showed substantial improvements across all metrics, with a 5.7% increase in mAP50, 11.3% in mAP75, and 12.3% in mAP50-95. This improvement demonstrates the effectiveness of our dataset enhancement techniques in creating a more robust ANPR system capable of handling diverse and challenging real-world scenarios.

5.4. Evaluation of Pre-Processing Techniques

We explored various pre-processing techniques to further enhance the ANPR system’s performance. However, our findings revealed that these techniques did not significantly improve the overall accuracy of the YOLOv8 model. Specifically, the observed decrease in mAP as the number of GMM cluster centers increases can be attributed to the over-segmentation of image features. While techniques like GMM pre-processing can enhance certain image characteristics, they may also introduce artifacts or oversimplify complex patterns that are important for LP recognition. The finer segmentation from increased GMM clusters may lead to a loss of contextual information that the YOLO model relies on for accurate detection, as shown in Figure 16.

The results presented in

Table 4. Impact of pre-processing techniques on model performance.

Pre-Processing Method	mAP50
No Pre-processing	0.87
Edge Detection	0.86
K-mean Thresholding	0.87
DBSCAN	0.85
GMM (6 clusters)	0.86
GMM (12 clusters)	0.85
GMM (24 clusters)	0.84

highlight a critical finding: modern DL models like YOLOv8 can learn robust features directly from raw input images. The consistent performance of the model without pre-processing (mAP50 of 0.87) suggests that YOLOv8’s internal feature extraction mechanisms are highly effective at the ANPR task. This observation aligns with recent trends in computer vision, where end-to-end learning on raw inputs often outperforms traditional pipeline approaches with separate pre-processing steps.

While pre-processing techniques may not enhance performance in this context, their exploration remains valuable for understanding the model’s behavior and the nature of the ANPR problem. These findings inform future research directions, suggesting that efforts might focus better on data quality, augmentation strategies, and model architecture improvements rather than on complex pre-processing pipelines. This finding is significant as it simplifies the ANPR pipeline and reduces computational overhead.

5.5. System Performance in Real-World Deployment

The deployment of our ANPR system using a Raspberry Pi 4B as the edge device and a high-performance server for processing demonstrated excellent real-world performance.

5.5.1. Processing Speed

Table 5. Average processing times for ANPR pipeline stages.

Stage	Average Time (ms)
Image Capture (Raspberry Pi)	50
Image Transmission	100
LP Detection	6.2
Number Recognition	9.7
Total Processing Time	166

shows the average processing times for different stages of the ANPR pipeline.

We recorded the average processing times during the real-world deployment of our system using a Raspberry Pi 4B as the edge device and a high-performance server for processing. These are based on the empirical measurements that reflect typical conditions in the deployment environment, ensuring the times represent the system’s operational performance. Additionally, the measurements were taken over a sample of more than one hundred (100) consecutive ANPR operations during our tests to ensure a representative benchmark. The total processing time of 166 ms represents the end-to-end latency from image capture to result communication, demonstrating the system’s capability for real-time ANPR in practical deployment scenarios. This implies that the ANPR core process, i.e., LP detection and number recognition, takes about 15.9 ms, which is equivalent to 62.89 FPS.

5.5.2. Accuracy in Various Conditions

We tested the system under different environmental conditions to assess its robustness.

Table 6. System accuracy under various environmental conditions.

Condition	Accuracy (%)
Daylight, Clear Weather	98.5
Night-time	95.2
Rainy Conditions	93.7
Partial Occlusion	91.8

summarizes these findings.

The variations in accuracy across different environmental conditions can be attributed to several factors:

• Daylight, clear weather: This represents optimal conditions with good lighting and clear visibility, allowing for high-quality image capture and easy LP detection.
• Night-time: Reduced lighting conditions can lead to lower image quality and contrast, making LP detection and OCR more challenging. However, our dataset augmentation techniques simulating low-light conditions help maintain relatively high accuracy.
• Rainy conditions: Rain can cause image blur, reflections, and reduced contrast. These factors can obscure LP details and boundaries, increasing the difficulty of accurate detection and recognition.
• Partial occlusion: When parts of the LP are obscured, the model must infer missing information, which can lead to increased errors. The relatively high accuracy in this challenging scenario demonstrates the robustness of our approach.

The system’s ability to maintain over 90% accuracy across these diverse conditions underscores the effectiveness of our comprehensive training approach and dataset augmentation strategies.

5.6. Comparative Analysis with Existing Works and Discussion

To contextualize our results within the broader field of ANPR research, we conducted a comparative analysis with recent, relevant studies.

Table 7. Comparison of our ANPR system with existing works.

Study	Model	Accuracy	Processing Time	Dataset Size	Environmental Conditions
[37]	YOLOv8, Faster R-CNN	68.6% 51.0%	-	-	-
[38]	YOLOv8, ResNet-50	98.6% 97.8%	-	1300	Day and night
[43]	YOLOv3	90.0%	-	2500	-
[44]	YOLOv4	70.38%	2700 ms	3000	Day
[45]	YOLOv4, R-CNN	96.0% 87.8%	-	150	Day
[46]	YOLOv3, VGG16	99.8% 99.1%	-	7035	Day
[47]	YOLOv4	90.3%	-	Five public datasets	-
[48]	RetinaNet	89.38%	143 ms	4500	Various
[49]	CNN (sliding window)	87.36% (normal) 44.93% (noisy)	1.06 ms/character	2518	Various (normal, blur, and night)
Our Work	YOLOv8s	98.5%	Overall: 166 ms ANPR: 62.89 FPS	2432	Various (day, night, rain, occlusions)

presents a summary of this comparison, showcasing key performance metrics across different ANPR approaches.

While our YOLOv8s-based ANPR system demonstrates competitive performance across key metrics, it is important to note the challenges with directly comparing different ANPR systems. Notably, our system achieves one of the highest accuracy rates (98.5%) among the compared studies, while also maintaining the fastest processing times (166 ms for the overall process and 62.89 FPS for LP detection and character recognition). Now, observing, for instance, the high accuracy achieved by [38] using YOLOv8 with a smaller dataset highlights the importance of considering each study’s specific contexts and challenges. Our system was developed and tested in Qatar’s dry and dusty subtropical desert climate, which features challenging conditions that impact visibility, including dust and intense sunlight, which may not be present in other datasets. Additionally, our reported processing time of 166 ms encompasses the entire pipeline, including image capture, transmission, and processing, providing a comprehensive measure of real-world performance.

The robustness of our system across various environmental conditions, particularly its maintained accuracy of over 93% in adverse scenarios, sets it apart from several studies that focused primarily on daytime conditions. This adaptability is crucial given Qatar’s diverse climate and lighting conditions.

Furthermore, our dataset enhancement techniques, which simulated adverse conditions, likely contributed to the system’s robust performance across various scenarios. While our dataset size (2432 images) is comparable to most studies in the comparison, the quality and diversity of our dataset, enhanced through simulation techniques, may have played a crucial role in achieving superior overall performance in varied real-world conditions.

The use of YOLOv8, a more recent iteration compared to the models used in other studies, potentially contributed to our system’s improved performance. YOLOv8’s architectural improvements over its predecessors, combined with our optimization for Qatari LPs, resulted in a more efficient and accurate ANPR system.

However, it is important to note that direct comparisons between ANPR systems can be challenging due to variations in datasets, specific environmental conditions, and the unique characteristics of LPs in different regions. Our system’s specialization for Qatari LPs, while a strength in our context, may limit direct comparability with systems designed for other regions.

The results of our study demonstrate the effectiveness of our YOLOv8-based ANPR system in addressing the unique challenges posed by Qatar’s LP recognition needs. The performance of the YOLOv8s model, achieving mAP50 values of 0.994 for LP detection and 0.986 for number recognition, represents a significant advancement in ANPR technology. This high level of accuracy, combined with the model’s computational efficiency, makes it particularly suitable for real-time applications in traffic management and vehicle identification.

The selection of the YOLOv8s model over its larger counterparts (m, l, and x) highlights an important finding in the trade-off between model complexity and performance. While larger models showed marginally better accuracy, the small model’s ability to maintain high precision with significantly reduced inference time (6.2 ms for plate detection and 9.7 ms for number recognition) demonstrates its superior suitability for real-world deployment. This balance is crucial in ANPR systems, where real-time processing is often as important as accuracy.

Our dataset enhancement technique, which involved simulating various adverse conditions, proved to be a key factor in improving the model’s robustness. The significant improvement in performance on low-quality LPs (5.7% increase in mAP50, 11.3% in mAP75, and 12.3% in mAP50-95) underscores the importance of training on diverse and challenging scenarios. This approach effectively bridges the gap between controlled training environments and the variability encountered in real-world deployments. It suggests that future ANPR systems could benefit greatly from similar data augmentation strategies, especially in regions with diverse environmental conditions.

The exploration of various pre-processing techniques yielded an interesting and somewhat counterintuitive result. The lack of substantial improvements from additional pre-processing steps, such as edge detection, k-mean thresholding, DBSCAN, and Gaussian mixture models, suggests that modern DL models like YOLOv8 are capable of learning robust features directly from raw input data. This finding has important implications for the design of ANPR pipelines, as it indicates that complex pre-processing stages may be unnecessary, thereby reducing system complexity and computational requirements. However, it is worth noting that these pre-processing techniques might still prove valuable in specific scenarios or with different model architectures, and further investigations in varied contexts could provide additional insights.

The real-world deployment results, showcasing high accuracy across different environmental conditions, validate the effectiveness of our distributed architecture. By leveraging the Raspberry Pi for image capture and a high-performance server for processing, we have created a system that is both flexible in deployment and powerful in execution. The total processing time of 166 ms demonstrates the system’s capability for real-time ANPR and is suitable for high-speed traffic monitoring and vehicle access control applications. This architecture also provides a scalable solution that can be easily adapted to different operational scales, from single-point installations to city-wide networks.

The system’s performance under various environmental conditions, maintaining over 93% accuracy even in challenging scenarios like night-time and rainy conditions, is particularly noteworthy. This robustness is crucial for the reliable operation of ANPR systems in Qatar’s diverse climate conditions. However, the slight degradation in performance under adverse conditions (from 98.5% in clear daylight to 93.7% in rainy conditions) indicates that there is still room for improvement, particularly in extreme weather scenarios. Overall, our ANPR system demonstrates state-of-the-art performance in the context of Qatar’s traffic management needs, providing a solid foundation for future developments in intelligent transportation systems.

While our research has made substantial progress, several avenues for future work and improvement have been identified. Extended environmental testing in extreme weather conditions, such as sandstorms or the intense heat characteristic of Qatar’s climate, could further enhance the system’s robustness. Expanding the system’s capabilities to detect and track multiple LPs in crowded traffic scenarios simultaneously could improve its utility in busy urban environments. Integration with broader traffic management and smart city infrastructures could open up new applications, including real-time traffic flow analysis, parking management, and automated toll collection systems.

Future work could also focus on optimizing the ANPR model for edge or cloud deployment, reducing reliance on server-side processing. This may involve model compression techniques, quantization, or exploring edge-specific AI architectures to enable more direct processing on devices like the Raspberry Pi. Implementing adaptive learning mechanisms that allow the system to continuously improve its performance based on real-world data could enhance the long-term accuracy and adaptability to changing conditions.

Exploring the adaptability of our system to LPs from different regions or countries could broaden its applicability and may potentially involve the development of transfer learning techniques to quickly adapt the model to new LP styles with minimal additional training data. As ANPR systems handle sensitive data, future work should also focus on enhancing data privacy and security measures, including developing anonymization techniques for stored plate data and implementing robust encryption for data transmission.

Expanding the system’s capabilities beyond LP recognition to include vehicle make and model identification or the detection of vehicle modifications could provide additional value for law enforcement and traffic management applications. Investigating ways to reduce the energy consumption of the ANPR system, particularly in the context of 24/7 operation, aligns with global sustainability goals and could involve optimizing hardware configuration and developing more energy-efficient processing algorithms.

The implications of this advanced ANPR system extend far beyond its immediate technological applications. By providing accurate, real-time data on traffic patterns and vehicle movements, our system has the potential to revolutionize urban planning and traffic management policies in Qatar. City planners and policymakers can utilize these data to make informed decisions about road infrastructure development, public transportation routes, and traffic flow optimization. This data-driven approach can lead to more efficient use of urban space, reduce traffic congestion, and, ultimately, lower vehicle carbon emissions.

Moreover, the system’s ability to operate effectively in Qatar’s challenging climate conditions contributes significantly to the city’s resilience. As urban areas face increasing pressure from climate change and rapid population growth, technologies that can adapt to and perform under various environmental conditions become crucial. Our ANPR system, with its robust performance across different weather scenarios, exemplifies the kind of resilient infrastructure needed for sustainable urban development.

6. Conclusions

In this work, we successfully developed and implemented an advanced ANPR system tailored explicitly for Qatar’s diverse LP landscape and challenging environmental conditions. The study made significant contributions to the field of ANPR by demonstrating the effectiveness of the YOLOv8 model, particularly the YOLOv8s variant, in achieving high accuracy and computational efficiency for both the LP detection and number recognition tasks. The model’s performance, with mAP50 values exceeding 0.98 for LP detection and 0.986 for number recognition, sets a new benchmark in ANPR systems.

Our innovative approach to dataset enhancement, involving the simulation of adverse conditions such as deterioration, low contrast, and poor focus, significantly improved the model’s robustness. This technique resulted in a 12.3% improvement in mAP50-95, enhancing the system’s ability to handle challenging real-world scenarios. The integration of edge computing using a Raspberry Pi with server-side processing demonstrated an effective solution for real-time ANPR applications, achieving an average overall processing time of 166 ms per image and 62.89 FPS for ANPR. This makes it suitable for high-speed traffic monitoring and access control systems.

Extensive testing under various environmental conditions, including night-time and rainy scenarios, validated the system’s reliability: maintaining over 93% accuracy in adverse conditions. The research also provided valuable insights into the limited impact of additional pre-processing techniques on modern DL models like YOLOv8, streamlining the ANPR pipeline and reducing computational overhead.

These achievements collectively represent a significant advancement in ANPR technology, particularly in the context of Qatar’s unique traffic management needs. The system’s ability to maintain high accuracy across diverse conditions while operating in real-time demonstrates its potential for wide-scale deployment in smart city initiatives and intelligent transportation systems.

This work offers more than just technological benefits; it provides real-time data that can transform Qatar’s urban planning and traffic management. By enabling data-driven decisions, it supports efficient infrastructure development, reduces congestion, and lowers vehicle emissions. Additionally, the system’s resilience in Qatar’s challenging climate underscores its value as a crucial component of sustainable urban development.

Despite the significant advancements presented in this study, there are several critical points and potential areas for future improvement in our ANPR system. One critical point is the slight degradation in performance under extreme weather conditions, such as heavy rain or sandstorms, which are characteristic of Qatar’s climate. Future work could focus on further enhancing the system’s robustness in these extreme conditions, possibly through more advanced data augmentation techniques or by incorporating weather-specific models.

Another area for improvement is the system’s current reliance on server-side processing for complex computations. Future iterations could explore edge computing solutions to reduce latency and enhance real-time processing capabilities. This could involve developing more efficient, lightweight models that can run directly on edge devices without compromising accuracy.

Additionally, while our system performs well with Qatari LPs, its adaptability to plates from other regions could be improved. Future research could focus on developing transfer learning techniques to quickly adapt the model to new LP styles with minimal additional training data.

Lastly, as ANPR systems handle sensitive data, future work should also focus on enhancing data privacy and security measures. This could include developing anonymization techniques for stored LP data and implementing more robust encryption for data transmission between edge devices and servers.