A Critical Appraisal of Various Implementation Approaches for Real-time Pothole Anomaly Detection: Toward Safer Roads in Developing Nations ^†

Abstract

Road infrastructure is essential to national security and growth. Potholes on the road surface cause accidents and costly automotive damage. Novel technology that detects potholes and alerts drivers in real-time may address this challenge. These approaches can improve road safety and lower vehicle maintenance cost in resource-constrained developing nations. This study reviews deep learning and sensor-based pothole detection approaches. An analysis shows that deep learning computer-vision-based algorithms are the most accurate, but computational and economic constraints limit their use in developing nations like Nigeria. Meanwhile, the sensor-based solutions are cost-effective and can be utilized in developing nations for pothole detection.

1. Introduction

Road infrastructure is an important part of modern society since it facilitates transit, commerce, and general economic development [1,2]. The condition and maintenance of roads, on the other hand, can have a substantial impact on safety and efficiency. Road maintenance authorities and motorists have long struggled with potholes, which are prevalent in developing nations [3]. Potholes not only cause car damage, but they also pose a significant risk to road safety, frequently resulting in accidents and injuries [4]. Pothole detection and maintenance have traditionally relied on manual inspections, which can be time-consuming, costly, and prone to human mistake [5]. However, the introduction of deep learning techniques in computer vision and artificial intelligence (AI) and improved sensor technology like LiDAR (Light Detection and Ranging) sensors that emit laser pulses or light beams and measure the time it takes them to bounce off objects and return to the sensor [6], and ultrasonic sensors that measure distances by measuring the time it takes ultrasonic pulses to bounce off objects and return to the sensor [7], have revolutionized the road surface condition monitoring approach and specifically real-time pothole detection and notification systems.

These two approaches have received a lot of interest from academia and industry for real-time road anomaly monitoring and alerting systems, especially for potholes. Thus, this article examines the use of deep learning, LiDAR, ultrasonic sensors, and other Internet of Things (IoT) devices for pothole detection, highlighting their benefits, potential impact on road maintenance and safety, and potential adoption in developing countries like Nigeria. Deep learning techniques, notably convolutional neural networks (CNNs), have gained widespread acceptance and demonstrated efficacy in a wide range of computer vision applications such as image categorization, object recognition, and segmentation [8]. CNNs are an excellent alternative for detecting potholes in road images due to their adaptability and agility [9,10]. The CNNs have been widely used for pothole detection, attaining high precision while dramatically lowering false positives when compared to traditional machine learning approaches [11]. This capability enables real-time detection and alerts drivers or maintenance teams, enabling timely responses to road hazards. In addition, the video-based pothole identification showed the promise of deep learning systems beyond static images [12].

Deep learning models are adaptable to changing road and lighting conditions. They can learn and generalize from a variety of datasets, making them adaptable to a variety of contexts [9]. While deep learning holds enormous promise for pothole detections, challenges persist. Data availability, model complexity, and the need for significant computational resources are some of the hurdles that researchers and practitioners face. Deep learning models that are accurate and efficient require large labelled datasets, which can be difficult to obtain. Model training and deployment necessitate robust computational infrastructure, which can have an impact on model deployment in developing nations.

Similarly, LiDAR technology has found extensive uses in a variety of industries, including agriculture, forestry, urban planning, and, most importantly, transportation infrastructure management [13]. One of the primary benefits of LiDAR technology in pothole detection is its capacity to acquire very precise and accurate spatial data [14]. LiDAR sensors emit laser beams that bounce off things, including road surfaces. LiDAR systems may generate high-resolution elevation maps of the road by measuring the time it takes for these laser pulses to return [14]. LiDAR is reliable for day and night operations since it works well in various lighting conditions. This adaptability provides continuous data collection regardless of climatic conditions for real-time road condition monitoring. LiDAR-based pothole identification involves installing sensors on vehicles and driving tests to collect point cloud data, which a pre-developed algorithm can use to identify road faults like potholes. Road maintenance officials and drivers receive quick alerts for potholes, enabling rapid response and repair. LiDAR’s precise depth and spatial information make pothole detection accurate and efficient, and its wide coverage area provides dependable detection.

Consequently, this article studies and analyses these two prominent ways for detecting pothole anomalies, as well as determining their feasibility for implementation in automobiles to aid drivers in developing countries such as Nigeria. Thus, this serves as the major contribution of this survey paper. The rest of the paper is structured as follows: Section 2 evaluates deep learning techniques for pothole detection, highlighting their pros and cons. Section 3 discusses LiDAR and ultrasonic sensors for real-time pothole identification and their pros and cons. Meanwhile, conclusions are drawn in Section 4.

2. Deep Learning Approaches for Pothole Detection

A road anomaly identification system that incorporates the use of a convolutional neural network (CNN) and visual-transformer-based approach for detecting road anomalies, notably potholes, cracks, and alligators, was presented in [11]. The EfficientDet, YOLOv4, YOLOX, and Swin Transformer deep learning models were chosen for their investigations due to their distinct properties. Each model was trained using annotated Canadian road deterioration condition datasets including around 27,298 images of potholes, cracks, and alligators, respectively. The system’s performance was assessed using a confusion matrix as well as metrics such as mean average precision (mAP) and intersection of union (IoU), as given in Equations (1) and (2), respectively. These measurements showed that the Swin Transformer model performed better, with 74% detection accuracy and a processing speed of 42 frames per second. Despite the fact that it requires a large processing resource to train each of the deep learning models, real-time deployment of the proposed model for detection of road anomalies was not investigated.

$$mAP=\frac{1}{\left|Q_R\right|}{\sum }_{q\in Q_R}AP\left(q\right)$$

(1)

where $Q_R$ is the number of validations set and $AP$ is the average precision.

$$IoU=\frac{area\left(B_{p }\cap B_{gt}\right)}{area\left(B_p \cup { B}_{gt}\right)}$$

(2)

where $B_p$ and ${ B}_{gt}$ are the predicted and ground truth bounding box.

A method that includes the deployment of a deep learning model for detecting road anomalies (potholes), notably You Only Look Once version3 (Yolov3), is presented in [15]. This required the use of 330 image datasets to train the model. The trained model was used for real-time detection by integrating a webcam for real-time road surface image data acquisition, a GPS module for real-time coordinate logging, and visualization of detected potholes using the Googlemap API. The testing results showed that the proposed system could achieve 90% accuracy and 65.05 mAP. However, limited datasets for deep learning model training affect the proposed approach’s performance. Similarly, a technique for detecting potholes that uses YOLOV3 and a Faster-Region-based Convolutional Neural Network (F-RCNN) was presented in [16]. This involves training each deep learning model with road surface image datasets and extracting pothole images from road network recordings. With 90% detection accuracy, the proposed technique was shown to be effective. A real-time deployment method was investigated, but the system cannot localize road anomalies. In developing nations, the need for GPUs and uninterrupted electricity during training may hinder its implementation.

An MVGG16, a modified VGG16 deep learning architecture for pothole detection, was presented in [17]. This involved changing dilation rates and removing several convolution layers to improve MVGG16 training performance and reduce computational cost. The suggested MVGG16 was compared to YOLOv5, ResNet101, ResNet50, and VGG16 and used as a backbone network for a quicker R-CNN. Compared to VGG-16, MobilNetV2, and InceptionV3, YOLOv5 significantly improves performance accuracy, mean precision, and inference time when used as the backbone of a quicker R-CNN for real-time pothole identification. Thus, MVGG16 balances pothole detecting speed and accuracy. However, the ambient meteorological circumstances in which image data are taken affect model training and system performance. Computational resource constraints may limit the application of these models in developing nations due to their initial training requirements. We note that a similar approach that uses YOLOv5 and FRCNN was also explored for pothole detection in [18]. Results showed a considerable performance accuracy when deployed for real-time detection on an India road network.

In [19], a solution that combined the use of an accelerometer and an ultrasonic sensor for real-time pothole detection based on a deep convolutional neural network (CNN-DL) was described. The sensor output was processed in real-time for the detection of potholes and humps. The GPS system embedded in their design aided in obtaining the corresponding coordinate location of potholes and thereby alerting appropriate agencies. The experimental findings illustrate the usefulness of the suggested approach for detecting potholes and humps. Nonetheless, the suitable positioning of such sensors to keep them from being damaged by an environmental effect as well as other interferences such as the intensity of sunlight that influences the operation of the ultrasonic sensor remained a challenge. A similar approach based on an advanced IoT-based technology that utilizes a YOLOv7 deep learning model and ultrasonic sensor as a two-stage approach for pothole detection toward ensuring the accuracy of the proposed system was reported in [20]. The approach shows potential for a possible adoption and deployment for use in developing nations.

Furthermore, in [21], a CNN-based model, especially YOLOV3, was proposed for the classification of various road types, including paved, unpaved, and asphalt roads. In addition, the suggested model was trained to detect the existence of potholes on classed asphalt images with an accuracy of 96%, while the road classification achieved an accuracy of 88%. Also, an approach that incorporated a CNN and web-based application embedded in an unmanned area vehicle for road anomaly detection, viz. potholes, cracks, and other defects, was reported in [22]. Despite showing potential for the successful detection of pothole anomalies and reporting instances to appropriate authorities for planning maintenance, its suitability for deployment in developing nations is difficult, where the internet network facilities are not yet optimal for the operations of such technology.

For real-time pothole detection on Indian roads, a transfer learning-based technique called a Faster-Region-based Convolutional Neural Network (F-RCNN) and Inception-V2 was created [23]. This included connecting the established model processing device with a web camera positioned in front of a car to acquire a live stream of road surface data, analyze them in real-time, and detect the existence of potholes. Nonetheless, the experimental findings demonstrated that the proposed technique was capable of detecting the underlying anomaly. However, the localization of the identified pothole anomaly was not incorporated in their design, which is critical to assisting road maintenance in developing countries. These documented findings are summarized in

Table 1. Deep Learning Approaches for Pothole Detection.

Paper	Goal	Method	Limitation	Performance Accuracy	Real-Time Deployment Capability
[11]	Road anomaly detection using CNN and visual transformer	EfficientDet, YOLOv4, YOLOX, and Swin Transformer deep learning models were used to detect road anomalies.	System training requires lots of computing resources	Swin Transformers achieve 74% accuracy	NO
[15]	YOLO3-based pothole detection	A trained model with real-time detection using a webcam for road surface image data gathering, a GPS module for coordinate logging, and the Googlemap API to visualize potholes.	Small training dataset used by the model serves as a drawback	90% accuracy and 65.05 mAP was achieved	YES
[16]	Used YOLOV3 and Faster-Region-based Convolutional Neural Network (F-RCNN)	This requires training each deep learning model with road surface image datasets and extracting pothole images from road network recordings.	The system could not localize detected road anomalies	90% detection accuracy was achieved	YES
[17]	Article highlights a modified VGG16 deep learning architecture for pothole detection	Achieved by changing dilation rates and removing several convolution layers to improve MVGG16 training performance.	Requires a huge computational resource, which can affect the model training and system performance	Outperforms other models in terms of performance accuracy, mean precision, and inference time when used as the backbone of a quicker R-CNN	YES
[18]	Pothole detection that uses YOLOv5 and FRCNN	Similar methodology as presented in [17].	Considered only an Indian road network	No measurable value though satisfactory results achieved	YES
[19]	A deep convolutional neural network (CNN-DL) that combines an accelerometer and an ultrasonic sensor for real-time pothole detection	Sensor output was processed in real-time to detect potholes and humps. The GPS system in their design helped localize the detected potholes.	Suitable positioning of the sensors was a key challenge	Only experimental demonstration with no measurable value as evidence	YES
[21]	A CNN-based model using YOLOV3 that classifies various road types	Proper classification of the road types into paved, unpaved, and asphalt roads.	The system could not be ascertained to aid real-time detection and deployment especially in developing countries	96% accuracy for classification of potholes based on asphalt. Other road classifications have 88% accuracy	NO
[22]	CNN-based road anomaly detection with a UAV-based web application	Incorporate a trained model and a web-based application in a UAV deployed for monitoring road surface conditions and detecting road anomalies.	The system may experience malfunctions when deployed in developing nations with limited internet access	The system was successfully able to detect different road anomalies and log such information in the design web application	YES
[23]	A transfer learning-based technique called Faster-Region-based CNN and Inception-V2 was considered for Indian roads	The model was connected to a processing device with a web camera positioned in front of the vehicle to acquire live stream of road surface data.	Localization of identified potholes was not considered in their design	No means to ascertain measured accuracy	YES

, while highlighting the method, strength, limitation, and possibility of real-time deployment for each of the manuscripts reviewed.

3. Exploration of LiDAR- and Ultrasonic-Sensor-Based Approaches for Pothole Detection

This section evaluated various contemporary methodologies that investigated the use of LiDAR and ultrasonic sensors for pothole detection. We observe that, despite the limited recorded literature on the application of this technology for real-time pothole detection, it nevertheless offers a reasonable promise for implementation in developing countries such as Nigeria. This is due to its low cost and convenience of implementation to assist drivers in navigating an anomalous road network, thus ensuring the safety of lives and properties.

An automated pothole anomaly detection system that makes use of an ultrasonic sensor, a GPS module, and a driver alerting system was reported in [24]. The ultrasonic sensor assessed the road surface and detected potholes, while an accelerometer sensor incorporated in the system determined the depth of the perceived pothole, notifying drivers and sending an email warning to approved users. Experiment results show that it is suitable for pothole detection with about 90% performance accuracy at speeds ranging from 10 to 50 km/h. Though suited for deployment in developing nations, new strategies that could minimize the impact of environmental conditions on the functioning of the system when deployed for use should be considered. Similarly, ref. [25] reported a user-friendly approach that incorporates the use of an ultrasonic sensor and a Raspberry Pi as a processor for real-time pothole anomaly detection. Potholes were detected using the timing difference between the radiated and reflected signal pulses. The experimental results show that the proposed approach for real-time detection was effective. However, the system’s inability to localize the identified pothole and save such information on a remote database or onboard to allow appropriate authorities to plan and prioritize maintenance is a drawback of the developed system.

A 2D LiDAR sensor technique for detecting the ground, barriers, and potholes was reported in [26]. This requires the use of Robotic Operative System (ROS) nodes to facilitate easy modification of the mobile robots with sensors connected. When the sensor’s angle of inclination to the horizontal axis is at 15°, data are obtained. The sensor’s position requires a height of 55 cm. Furthermore, a 2m distance from the robot is required for data collection. The data were converted from polar to cartesian using a point cloud. Line detection finds barriers in segmented point clouds. The Euclidean distance between points helps identify lines. Ground lines were ‘green’, impediments were ‘red’, and potholes were ‘yellow,’ according to the experimental results. Due to the indoor trial, obstructions and potholes were not quite as prevalent as expected. Additionally, sensor usage caused testing errors. Thus, this hindered real-time deployment.

Similarly, ref. [14] proposed an automatic pothole detection method based on a 2D LiDAR sensor. This entailed the installation of two 2D LiDAR sensors separated by one meter, which housed a mobile camera capable of scanning the road surface across a 4 m broad area. Scanning occurs 30 times to aid in reliable data capture, and the camera unit records and retains the information scanned. To aid in pothole detection, four processes were performed: filtering, clustering, line extraction, and a gradient of data function. MATLAB was used to simulate estimated pothole information such as breadth and depth and compare it to actual potholes. The results revealed successful detection with low error rates. Nevertheless, in a 2D LiDAR simulation, the pothole was moved farther from the road center.

A revolutionary method of detecting and filling potholes utilizing an ultrasonic sensor technology was proposed in [27]. Canny-edge detection and image processing techniques were utilized to classify the various road types. An ultrasonic sensor measures the depth and range of potholes, and the findings are shown on a mobile device. The Canny-edge approach was shown to be 87.5% accurate. However, the proposed approach did not consider real-time identification and localization of potholes. Similarly, an Internet of Things (IOT)-based method for detecting potholes was proposed in [28]. The suggested concept includes an interface where users can register their information about the path they want to take. The database part provides the number of potholes that are expected to be met along the chosen path, as well as their geolocations. A robot, ultrasonic sensor, and ESP-8266 module are included with the sole purpose of detecting potholes retrieved from the android application in real-time. Despite the efficacy of the method for pothole detection and alerting, its performance cannot be quantified.

Table 2. Exploration of LiDAR- and Ultrasonic-Sensor-based Approaches for Pothole Detection.

Paper	Goal	Method	Limitation	Performance Accuracy	Real-Time Deployment Capability
[24]	An automated pothole anomaly detection system that uses a GPS module and an ultrasonic sensor	The accelerometer measured the depth of the potholes. However, the ultrasonic sensor accessed the road surface.	Requires new strategies to minimize environmental impact(s)	90% accuracy	YES
[25]	Real-time pothole detection that incorporates ultrasonic sensor and Raspberry Pi	Detection occurs via timing difference between radiates and reflected signal pulses.	Portrayed some challenges in localization of identified potholes	No quantitative value. However, effective in real-time detection	YES
[26]	Has potential of detecting ground, barriers, and potholes	Produces a 3D map from a downward suspended 2D LiDAR sensor using Robotic Operative System (ROS) software.	An indoor experimental trial that did not employ mobile robots for the sensors. Furthermore, the system did not consider obstructions and potholes	Could not ascertain measurable accuracy values	NO
[14]	An automated means of pothole detection based on 2D LiDAR sensor	Implemented through the installation of two 2D LiDAR sensors separated by one meter, which housed a mobile camera capable of scanning the road surface across a 4 m broad area.	The simulation revealed that potholes moved farther away from the center of the road during detection	No measured value. However, detection was with minimized error rates	YES
[27]	A new method of detecting and filling potholes utilizing an ultrasonic sensor technology. In addition, the approach makes use of Canny-edge detection and image processing to classify road types	An ultrasonic sensor measures the depth and range of potholes, and the findings are shown on a mobile device.	The approach was not implemented in real-time for proper identification and localization of potholes	The system performs with 87.5% accuracy using Canny-edge detection	NO
[28]	Presented a pothole detection system based on the Internet of Things (IOT)	The model employs an interface where users can register their information about the path they want to take. The database part provides the number of potholes that are expected to be met along the chosen path, as well as their geolocations.	Difficulty in quantifying the performance of the system	No clear means of measuring performance	YES

summarizes these different approaches that have explored the use of LiDAR and ultrasonic sensors for pothole detection.

4. Conclusions

This study provides a concise detailed assessment of modern deep learning algorithms, as well as the application of sensors such as LiDAR and ultrasonic sensors for real-time pothole detection on asphalt road surfaces. We note that, despite the reported performance of various deep learning methods with improved performance accuracy, the initial cost of computational resources such as GPU required for training the models may limit their widespread acceptability and suitability for deployment in developing countries such as Nigeria. Furthermore, despite environmental conditions that may occasionally affect the performance of sensor-based approaches for real-time pothole detection, they still have a higher potential for real-time pothole detection in developing countries due to their cost effectiveness and ease of implementation. Future research will investigate new methodologies that could improve sensor-based systems for monitoring road surface conditions in developing nations using IoT as well as possible deployment for monitoring rail line infrastructures.