Impact of Critical Situations on Autonomous Vehicles and Strategies for Improvement

Abstract

Recently, the development of autonomous vehicles (AVs) and intelligent driver assistance systems has drawn significant attention from the public. Despite these advancements, AVs may encounter critical situations in real-world scenarios that can lead to severe traffic accidents. This review paper investigated these critical scenarios, categorizing them under weather conditions, environmental factors, and infrastructure challenges. Factors such as attenuation and scattering severely influence the performance of sensors and AVs, which can be affected by rain, snow, fog, and sandstorms. GPS and sensor signals can be disturbed in urban canyons and forested regions, which pose vehicle localization and navigation problems. Both roadway infrastructure issues, like inadequate signage and poor road conditions, are major challenges to AV sensors and navigation systems. This paper presented a survey of existing technologies and methods that can be used to overcome these challenges, evaluating their effectiveness, and reviewing current research to improve AVs’ robustness and dependability under such critical situations. This systematic review compares the current state of sensor technologies, fusion techniques, and adaptive algorithms to highlight advances and identify continuing challenges for the field. The method involved categorizing sensor robustness, infrastructure adaptation, and algorithmic improvement progress. The results show promise for advancements in dynamic infrastructure and V2I systems but pose challenges to overcoming sensor failures in extreme weather and on non-maintained roads. Such results highlight the need for interdisciplinary collaboration and real-world validation. Moreover, the review presents future research lines to improve how AVs overcome environmental and infrastructural adversities. This review concludes with actionable recommendations for upgrading physical and digital infrastructures, adaptive sensors, and algorithmic upgrades. Such research is important for AV technology to remain in the zone of advancement and stability.

1. Introduction

The evolution of autonomous driving technology is becoming increasingly prevalent in our daily lives, driven by significant advancements in perception technologies and computational capabilities [1]. Just as the agricultural-to-industrial adaptation in the late 18th century was met with initial hesitation and uncertainty, there is current skepticism concerning AV safety, reliability, and societal impact. Reflecting on the Industrial Revolution, AV technologies will eventually integrate into our infrastructure, perhaps more gradually than proponents anticipate, reshaping our transportation systems. AVs are at the forefront of technological innovation, utilizing various sensors such as radar, lidar, and cameras to accurately detect and interpret their environment. This integration of cutting-edge technology empowers AVs to navigate complex scenarios safely and efficiently, marking a significant leap forward in the journey toward fully autonomous transportation. The driving automation is classified into six levels, from 0 (entirely manual driving) to Level 5 (complete automation in all scenarios).

Figure 1 illustrates the five levels’ automation level description and driver support. At Level 0, the human driver is fully in the loop and supplied with safety features such as emergency braking, blind spot warning, and lane warnings. Assistive features at Level 1 include lane keeping or adaptive cruise control, but the driver is fully engaged. At this level, Level 2, the driver can activate multiple assistive features simultaneously for convenience and support but is required to maintain full attention and control. Conditional automation, as it is called, can drive on its own in certain situations but gives way to the driver in uncertain circumstances. Systems exist to bring the vehicle to a safe stop in designated scenarios, and Level 4 is highly autonomous and capable of self-navigation. Finally, Level 5 is fully autonomous and requires no human input in any driving situation, potentially eliminating the need for traditional manual controls. In this review study, the focus will be explicitly on Level 5 automation.

With the growing interest in full autonomy, researchers can use these gaps to explore what is possible and start to put autonomous driving into traffic operations with the highest safety potential. Yet actual AVs may face dire traffic accidents. They include situations like heavy rain, snow, haze, fog, or dust when the sensors and cameras may be obscured by the situation, complex road environment, and infrastructure, unexpected debris on the road, or human drivers that are erratic or unpredictable. Furthermore, AVs need to be able to navigate construction zones, figure out how to manage system malfunctions, and respond to accidents or medical emergencies involving passengers. It is important to address these potential challenges to improve the safety and reliability of autonomous vehicles. Sensor technology has advanced in recent years, significantly improving its performance and resilience in various challenging conditions. Data from multiple sources, including radar, lidar, cameras, and GNSS, are combined using cutting-edge sensor fusion techniques to comprehensively view a vehicle’s surroundings [3,4,5]. Other researchers are developing advanced algorithms to enable AVs to interpret sensor data accurately, even if it is noisy or occluded [6,7,8,9]. Also, simulation, testing, and hardware enhancements are critical strategies. The critical situations recognize the impact of sensor performance, and research that will help improve perception is required to further develop AV technologies.

To date, some literature reviews have focused on evaluating the performance of standard sensors used in AVs under various weather conditions [2,10,11,12]. Rana et al. [11] critically reviewed the current state of advanced driver-assist systems (ADAS) in AVs, including their functions and deployment scenarios, and discussed the technological and infrastructural challenges. Vargas et al. [2] focused on the limitations of RADAR, LiDAR, ultrasonic sensors, cameras, and GNSS under heavy precipitation, suggesting potential hardware improvements. Yoneda et al. [12] analyzed research on automated driving technologies, focusing on recognizing and navigating under adverse weather conditions such as sun glare, rain, fog, and snow, and highlighted the difficulties these conditions pose for safe driving and market introduction.

Despite these efforts, to our knowledge, no paper has comprehensively covered all adverse weather phenomena and all standard AV sensors. Additionally, no paper has inclusively addressed all types of critical situations and the various solutions to improve AV performance in these scenarios. Therefore, this review aims to fill that gap by thoroughly examining the challenges and solutions investigated to improve AV sensor performance in critical situations including Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Generative Adversarial Networks (GANs), fusion and multi-modal networks, domain adaptation, unsupervised and semi-supervised learning, specialized segmentation and detection networks, simulation and testing, sensor fusion and multi-modal networks, data processing, and hardware enhancements. This comprehensive approach provides a holistic view of current research and future directions to enhance AV sensor performance under diverse and challenging conditions.

In the following sections of this study, Section 2 provides a detailed overview of radar technologies used in AVs, including their applications and limitations. Section 3 investigates the critical situations AVs encounter, such as adverse weather, complex environments, and infrastructure constraints. Section 4 concludes with future research directions to fill in the gaps discovered in AV technology performance and sensor reliability.

2. Sensor Technologies in AVs

AVs rely on various sensor technologies to sense and navigate their environment. These include radar, LiDAR, cameras, and global navigation satellite systems (GNSS), each of which contributes unique capabilities to ensure safe and efficient operations. In this section, we will explore these technologies in detail. Section 2.1 focuses on radar technology, which uses radio waves to detect objects and determine their distance. Section 2.2 discusses LiDAR, a laser system that provides accurate distance measurement. Section 2.3 covers the camera technologies necessary for visual perception and interpretation, and Section 2.4 examines GNSS, which enables real-time positioning and navigation. Figure 2 illustrates the basic principles of these sensors, and

Table 1. A comparative overview of sensors. “--” shows that the sensor performs moderately well under specific conditions, “✖” indicates poor performance under the given conditions, and “✔” represents optimal performance under the specified conditions.

Factors	Camera	LiDAR	RADAR	GNSS
Velocity	--	✖	✔	--
Object Detection	✖	✔	✔	--
Resolution	✔	--	✖	--
Range	--	✔	✔	✔
Distance Accuracy	--	✔	✔	✔
Lane Detection	✔	✔	✖	✖
Obstacle Edge Detection	✔	✔	✖	✖
Weather Conditions	✖	--	✔	✔
Situation Awareness	✔	✔	✔	✔
Cost	Low	High	Moderate	Moderate
Processing Time	Fast	Moderate	Fast	Moderate
Maintenance Requirements	Low	Moderate	Low	Low
Compatibility	High	Moderate	Moderate	High
Durability	Moderate	High	High	High
Spatial Coverage	Wide	Moderate	Wide	Wide

summarizes key research projects aimed at improving sensor performance.

2.1. Radar Technology Application

RADAR (Radio Detection and Ranging) technology plays a critical role in AVs by using radio waves to detect range, velocity, and moving objects within specific distances around the vehicle. This system consists of both a transmitter and a receiver: the transmitter sends out radio waves that reflect off objects, whether stationary or in motion, and the receiver captures these reflected signals (see Figure 2). By analyzing the time delay and frequency shift in the returned waves, the RADAR system accurately determines the position, speed, and direction of potential obstacles relative to the AV [13]. RADAR systems are tailored for distinct operational ranges: Short-Range RADAR (SRR 0.2 m), Medium-Range RADAR (MRR 30–80 m), and Long-Range RADAR (LRR 80–200 m) [2]. The LRR is particularly adept at detecting targets or objects ahead of the vehicle, while MRR and SRR are used for proximity applications like parking assistance and side view detection [3]. They operate at 24, 74, 77, and 79 GHz frequencies within the millimeter wave (MMW) spectrum. In alignment with American and European standards, the designated frequency band for automotive radar systems is around 77 GHz (λ ≈ 4 mm), commonly referred to as mm-wave radars [10], denoting their utilization of millimeter-wave frequencies.

2.2. LiDAR Technology Application

LiDAR, similar to RADAR, is a distance-measuring remote sensing technology. Unlike RADAR, which uses radio waves, LiDAR employs laser beams to measure distance by emitting an invisible laser light to human eyes on objects. The environment then reflects these laser beams to the device, where a photodetector captures the returned signals. Using the Time of Flight (ToF) method, LiDAR measures the duration required for the laser to travel to the target and back, allowing for precise distance calculation [14,15]. This feature allows LiDAR to generate detailed maps by capturing millions of data points. LiDAR also scans 360° continuously to create comprehensive 3D point clouds of its surroundings, with each pixel containing depth and motion speed information, providing a detailed view of the environment [16]. This operating principle is shown in Figure 2.

2.3. Camera Technology Application

Cameras are crucial in AVs and vital for observing and interpreting the vehicle’s surroundings. They are indispensable for recognizing pavement markings, identifying road signs, and spotting potential hazards like obstacles. Cameras are generally cost-effective, and when combined with appropriate software, they can distinguish between moving and stationary objects within their field of vision and produce high-resolution images of the environment [17]. This integration of cameras and advanced processing algorithms allows AVs to navigate complex environments with precision and reliability. Camera systems in AVs are primarily classified into two main types: visible (VIS) and infrared (IR). VIS cameras, monocular (single-camera), or binocular (dual-camera), operate in the 400 to 780 nm wavelength range, aligning with the spectrum visible to humans [2]. This allows them to capture a comprehensive worldview in conditions similar to human sight, making them invaluable for detailed environmental perception and navigation tasks. Stereo cameras and binocular cameras replicate the depth perception mechanism found in animals. They utilize the “disparity” between the two slightly different images captured by each camera to gauge depth, enhancing the vehicle’s ability to perceive and interact with its surroundings in 3D. This operating principle is shown in Figure 2.

IR cameras, operating in the infrared wavelength spectrum of 780 nm to 1 mm beyond what the human eye can see, can address several limitations associated with VIS cameras. They are particularly advantageous in low-light conditions, as they do not rely on visible light to capture images. IR cameras can be specialized into near-infrared (NIR) cameras, which cover the 780 nm to 3 µm range, and mid-infrared (MIR) cameras, ranging from 3 µm to 50 µm, also known as thermal cameras [18]. These adaptations allow IR cameras to serve a wide array of applications, providing enhanced capabilities such as improved night vision, warm body detection for pedestrians and animals [19,20,21], and the ability to see through obstructions that would typically impair the performance of VIS cameras.

The other popularly used camera in AVs is the fisheye, also widely used in AVs for its wide field of view for near-field sensing applications such as parking aid and traffic jam assistance. However, they can be used on AV systems with only four cameras, which can cover 360 degrees [22,23,24] and are a valuable way to have full environmental awareness. However, these cameras must be calibrated to correct optical distortions, such as pincushion, barrel, and moustache effects, to achieve precise object positioning and image accuracy [6,25].

Table 1 compares various sensors: Camera, LiDAR, RADAR, and GNSS, providing a comprehensive understanding of their strengths and limitations in advanced sensing applications. The table outlines key performance metrics: range, resolution, distance accuracy, velocity detection, and environmental adaptability. Cameras excel in color perception and lane detection, making them highly effective for tasks such as recognizing traffic lights and road markings. However, they are constrained by limited range and reduced performance in adverse weather conditions. LiDAR provides exceptional spatial resolution and distance accuracy, enabling precise object detection and classification, though it is associated with higher costs and energy demands. RADAR is well-suited for velocity detection and performs reliably in challenging weather conditions but lacks the resolution needed for fine-grained object classification. Additionally, factors such as energy consumption, cost, processing time, maintenance requirements, durability, spatial coverage, and compatibility further emphasize the trade-offs involved in sensor selection.

Building upon this comparison, infrared (IR) sensors emerge as an essential complement to these technologies, particularly in adverse environmental conditions. IR sensors operate effectively in fog, snow, and rain because they track thermal radiation from objects rather than the conventional visual light detection of traditional sensors like cameras or LiDAR. IR sensors offer valuable nighttime driving protection in addition to low-visibility services. IR sensors enhance detection performance in sensor fusion systems by delivering thermal imaging data to complement other sensing modalities. With the power to detect heat signatures, these components deliver exact identification of pedestrians alongside important objects and animals, thereby becoming essential in today’s autonomous vehicle technology [26].

2.4. Global Navigation Satellite System Technology Application

GNSS plays a crucial role in the operation of AVs, providing a real-time location, navigation, and timing of a vehicle, which is the core functionality for any navigation system combined with a digital map [27]. GNSS satellites transmit their current clock and position data via radio signals. Each GNSS signal consists of a carrier wave and a message modulated onto this carrier using a specific modulation process. Nowadays, GNSS mainly includes four typical satellite navigation systems: Global Positioning System (GPS), owned by the United States; Galileo, owned by the European Union; GLONASS, owned by Russia; and BeiDou, owned by China. GPS is the best-known among these systems, providing users with positioning, navigation, and timing (PNT) services. Currently, AVs use advanced techniques like differential global positioning systems (DGPS) and Real-Time Kinematic (RTK) positioning to improve the accuracy of corrections in positioning data [27]. The introduction of GNSS-RTN systems has enabled precise positioning over distances of 50–70 km from a base station, with the recommended spacing between reference stations not exceeding 70 km. By utilizing GNSS-RTN, systems can achieve positioning accuracy within the 1 to 5 cm range, in contrast to the 1 to 10 m accuracy typically seen without these advanced sensors [28]. In DGPS, the position error can be as low as 1 cm. With DGPS, positioning errors can be reduced to as little as 1 cm [27].

While each sensor type is suitable, the combination of LiDAR, RADAR, and camera data can vastly improve the perceptual accuracy and reliability of an AV. These systems combine the use of multiple sensors, allowing the system to account for the limitations of the individual sensors and form a robust detection and response framework for environmental variables. For example, cameras may fail in low visibility conditions, whereas LiDAR and RADAR can still measure distances and recognize obstacles. Together, these technologies allow autonomous vehicles to always know and accurately what they are around, making them safer and more efficient to operate. Research projects in Table 2 illustrate the importance of multi-sensor integration in advancing autonomous vehicle technologies. Through sensor fusion and integrated machine learning models, these projects demonstrate increased capabilities that can significantly improve accuracy and reliability in AVs.

Table 2. Research projects focused on sensor improvements in AVs.

Sensors	Sensor Applications	Improvement Method	Performance Evaluation	Key Findings	Reference
Thermal infrared camera and LiDAR	Object detection and classification in low-light and adverse weather conditions.	Sensor fusion calibrated with a 3D target.	Experiments conducted in day and night environments.	Improved object detection accuracy.	[5]
Millimeter wave radar	Real-time wide-area vehicle trajectory tracking.	Unlimited roadway tracking using millimeter wave radar.	Validation with Real-Time Kinematic (RTK) and UAV video data.	92% vehicle capture accuracy; position accuracy within 0.99 m of ground truth.	[29]
Traffic surveillance camera system	Detection, localization, and AI networking in autonomous vehicles.	Sensor fusion with AI networking capabilities.	Tested with various sensor combinations and machine learning models.	Improved detection accuracy and networking efficiency for autonomous operations.	[30]
High-resolution satellite images	Road detection in very high-resolution images	Semantic segmentation with attention blocks and hybrid loss functions for better edge detection.	Extensive testing on urban satellite images (Saudi Arabia and Massachusetts) using segmentation masks and edge detection metrics.	Significantly improved road detection and edge delineation with high accuracy in complex backgrounds	[31]
mm-wave radar	Recognition of vulnerable road users (pedestrians, cyclists) in intelligent transportation systems.	Shallow neural networks (CNN, RNN) for micro-Doppler signature analysis.	Tested recognition using CNN, RNN, and hybrid CNN-RNN on simulated datasets.	Achieved high recognition accuracy, enhancing road safety for vulnerable users.	[32]
LiDAR and camera	Road detection	LiDAR-camera fusion using fully convolutional neural networks (FCNs).	Evaluated on the KITTI road benchmark.	Achieved state-of-the-art MaxF score of 96.03%, outperforming single-sensor systems and ensuring robust detection in varying lighting conditions.	[4]
Deep Visible and Thermal Image Fusion	Enhanced pedestrian visibility in low-light and foggy conditions.	Learning-based fusion method producing RGB-like images with added informative details.	Qualitative and quantitative evaluations using no-reference quality metrics and human detection performance metrics, compared with existing fusion methods.	Outperformed existing methods, significantly improving pedestrian visibility and information quality while maintaining natural image appearance.	[33]
3D LiDAR + Monocular Camera	Urban road detection.	Inverse-depth induced fusion framework with IDA-FCNN and line scanning strategy using LiDAR’s 3D point cloud.	Evaluated on KITTI-Road benchmark with Conditional Random Field (CRF) for result fusion.	Achieved state-of-the-art road detection accuracy, significantly outperforming existing methods on the benchmark.	[34]
LiDAR and Monocular Camera	Pedestrian classification	Multimodal CNN leveraging LiDAR (depth and reflectance) and camera data fusion.	Evaluated on the KITTI Vision Benchmark Suite using binary classification for pedestrians, comparing early and late fusion strategies.	Achieved significant improvements in pedestrian classification accuracy through LiDAR-camera data fusion.	[35]
LIDAR and Vision (Camera)	Vehicle detection	PC-CNN framework fusing LiDAR point cloud and camera images via shared convolutional layers.	Evaluated on the KITTI dataset with 77.6% average recall for proposal generation and 89.4% average precision for car detection.	Achieved significant improvements in proposal accuracy and detection precision, highlighting its potential for real-time applications.	[36]
Multispectral (Visible and Thermal Cameras)	Pedestrian detection	Early and late deep fusion CNN architectures for visible and thermal data fusion.	Evaluated on the KAIST multispectral pedestrian detection benchmark, outperforming the ACF + T + THOG baseline with pre-trained late-fusion models.	Achieved superior detection performance, demonstrating robustness in varying lighting conditions.	[37]
LIDAR and RGB Cameras	Pedestrian detection	Fusion of LiDAR (up-sampled to a dense depth map) and RGB data using HHA.	Validated various fusion methods within CNN architectures using the KITTI pedestrian detection dataset.	Late fusion of RGB and HHA data at different CNN levels yielded the best results, especially when fine-tuned.	[38]
Dynamic Vision Sensor (DVS)	Computer vision in challenging scenarios.	Adaptive slicing of spatiotemporal event streams to reduce motion blur and information loss.	Evaluated on public and proprietary datasets with object information entropy deviation under 1%.	Achieved accurate, blur-free virtual frames, enhancing object recognition and tracking in dynamic scenes	[39]
GNSS, INS, Radar, Vision, LiDAR, Odometer	Vehicle navigation state estimation (position, velocity, attitude)	Multi-sensor integration with motion constraints (NHC, ZUPT, ZIHR) and radar-based feature matching.	Tightly coupled FMCW radar and IMU integration, tested for GNSS outage scenarios.	Improved navigation reliability by mitigating GNSS outages and correcting IMU drift, ensuring robust performance in diverse conditions.	[40]
Thermal cameras (LWIR range)	Pedestrian and cyclist detection in low-visibility conditions.	Deep neural network tailored for thermal imaging in variable lighting.	Evaluated on KAIST Pedestrian Benchmark dataset with paired RGB and thermal data.	Achieved an F1-score of 81.34%, significantly enhancing detection under challenging conditions where RGB systems struggle.	[41]

Table 2. Research projects focused on sensor improvements in AVs.

Sensors	Sensor Applications	Improvement Method	Performance Evaluation	Key Findings	Reference
Thermal infrared camera and LiDAR	Object detection and classification in low-light and adverse weather conditions.	Sensor fusion calibrated with a 3D target.	Experiments conducted in day and night environments.	Improved object detection accuracy.	[5]
Millimeter wave radar	Real-time wide-area vehicle trajectory tracking.	Unlimited roadway tracking using millimeter wave radar.	Validation with Real-Time Kinematic (RTK) and UAV video data.	92% vehicle capture accuracy; position accuracy within 0.99 m of ground truth.	[29]
Traffic surveillance camera system	Detection, localization, and AI networking in autonomous vehicles.	Sensor fusion with AI networking capabilities.	Tested with various sensor combinations and machine learning models.	Improved detection accuracy and networking efficiency for autonomous operations.	[30]
High-resolution satellite images	Road detection in very high-resolution images	Semantic segmentation with attention blocks and hybrid loss functions for better edge detection.	Extensive testing on urban satellite images (Saudi Arabia and Massachusetts) using segmentation masks and edge detection metrics.	Significantly improved road detection and edge delineation with high accuracy in complex backgrounds	[31]
mm-wave radar	Recognition of vulnerable road users (pedestrians, cyclists) in intelligent transportation systems.	Shallow neural networks (CNN, RNN) for micro-Doppler signature analysis.	Tested recognition using CNN, RNN, and hybrid CNN-RNN on simulated datasets.	Achieved high recognition accuracy, enhancing road safety for vulnerable users.	[32]
LiDAR and camera	Road detection	LiDAR-camera fusion using fully convolutional neural networks (FCNs).	Evaluated on the KITTI road benchmark.	Achieved state-of-the-art MaxF score of 96.03%, outperforming single-sensor systems and ensuring robust detection in varying lighting conditions.	[4]
Deep Visible and Thermal Image Fusion	Enhanced pedestrian visibility in low-light and foggy conditions.	Learning-based fusion method producing RGB-like images with added informative details.	Qualitative and quantitative evaluations using no-reference quality metrics and human detection performance metrics, compared with existing fusion methods.	Outperformed existing methods, significantly improving pedestrian visibility and information quality while maintaining natural image appearance.	[33]
3D LiDAR + Monocular Camera	Urban road detection.	Inverse-depth induced fusion framework with IDA-FCNN and line scanning strategy using LiDAR’s 3D point cloud.	Evaluated on KITTI-Road benchmark with Conditional Random Field (CRF) for result fusion.	Achieved state-of-the-art road detection accuracy, significantly outperforming existing methods on the benchmark.	[34]
LiDAR and Monocular Camera	Pedestrian classification	Multimodal CNN leveraging LiDAR (depth and reflectance) and camera data fusion.	Evaluated on the KITTI Vision Benchmark Suite using binary classification for pedestrians, comparing early and late fusion strategies.	Achieved significant improvements in pedestrian classification accuracy through LiDAR-camera data fusion.	[35]
LIDAR and Vision (Camera)	Vehicle detection	PC-CNN framework fusing LiDAR point cloud and camera images via shared convolutional layers.	Evaluated on the KITTI dataset with 77.6% average recall for proposal generation and 89.4% average precision for car detection.	Achieved significant improvements in proposal accuracy and detection precision, highlighting its potential for real-time applications.	[36]
Multispectral (Visible and Thermal Cameras)	Pedestrian detection	Early and late deep fusion CNN architectures for visible and thermal data fusion.	Evaluated on the KAIST multispectral pedestrian detection benchmark, outperforming the ACF + T + THOG baseline with pre-trained late-fusion models.	Achieved superior detection performance, demonstrating robustness in varying lighting conditions.	[37]
LIDAR and RGB Cameras	Pedestrian detection	Fusion of LiDAR (up-sampled to a dense depth map) and RGB data using HHA.	Validated various fusion methods within CNN architectures using the KITTI pedestrian detection dataset.	Late fusion of RGB and HHA data at different CNN levels yielded the best results, especially when fine-tuned.	[38]
Dynamic Vision Sensor (DVS)	Computer vision in challenging scenarios.	Adaptive slicing of spatiotemporal event streams to reduce motion blur and information loss.	Evaluated on public and proprietary datasets with object information entropy deviation under 1%.	Achieved accurate, blur-free virtual frames, enhancing object recognition and tracking in dynamic scenes	[39]
GNSS, INS, Radar, Vision, LiDAR, Odometer	Vehicle navigation state estimation (position, velocity, attitude)	Multi-sensor integration with motion constraints (NHC, ZUPT, ZIHR) and radar-based feature matching.	Tightly coupled FMCW radar and IMU integration, tested for GNSS outage scenarios.	Improved navigation reliability by mitigating GNSS outages and correcting IMU drift, ensuring robust performance in diverse conditions.	[40]
Thermal cameras (LWIR range)	Pedestrian and cyclist detection in low-visibility conditions.	Deep neural network tailored for thermal imaging in variable lighting.	Evaluated on KAIST Pedestrian Benchmark dataset with paired RGB and thermal data.	Achieved an F1-score of 81.34%, significantly enhancing detection under challenging conditions where RGB systems struggle.	[41]

3. Critical Situations for AVs

Critical situations for AVs arise from environmental and infrastructure problems, and Part 3 explores these situations in detail. Section 3.1 addresses adverse weather conditions such as rain, fog, and snow that can degrade sensor performance and navigation. Section 3.2 focuses on complex environments, including urban canyons and tunnels, where GPS signals can experience dysfunctionalities. Section 3.3 discusses road infrastructure conditions where a lack of signage, markings, and maintenance can cause navigation difficulties. These critical situations significantly challenge AV reliability and safety. Finally, Table 2, Table 3 and Table 4 summarize key research projects to improve critical situations.

3.1. Adverse Weather Conditions

Adverse weather conditions continue to pose a significant challenge for autonomous vehicles AVs despite continuous advancements in sensor technology. These conditions impact sensor performance, algorithm accuracy, and overall driving behavior, highlighting the importance of reliable operation under various environmental scenarios for automated driving on any road. This section summarizes the performance of different sensing technologies under various adverse weather conditions, including rain, fog, snow, hail, sun glare, lightning, dust, sandstorms, and contamination. The technical issues associated with each adverse condition are detailed in the following subsections.

3.1.1. Rain Effects

AVs are known to be severely degraded by rain, and it has been reported that rain is a major cause of radar signal attenuation. Rain backscatter was also identified by Wallace et al. [42] as a significant component of radar performance. The attenuation of electromagnetic (EM) energy by water droplets and the problem of backscattering or ‘rain clutter’, which can generate false signals and mask real targets from the radar’s detection capabilities [43], arises because raindrops are similar in size to the radar’s wavelength. Significant high-intensity rain backscattering is found on 77 GHz radars. Gourova et al. [44], however, note that at close distances of up to 30 m, the attenuation of the radar signal, which ranges from 0.0016 dB/m for light rain (1 mm/h) to 0.0032 dB/m for heavy rain (100 mm/h), has a negligible effect. Like Zhang et al. [10], they also pointed out that severe rainfall (150 mm/h) will reduce the detection range by up to 45% because of increased backscatter interference. Like LiDAR systems, rain impacts Mie scattering at 905 nm and 1550 nm wavelengths [2]. However, at the critical 250 m range used for AVs, LiDAR’s accuracy still needs to be improved, although greatly affected by extreme rainfall conditions [2,45]. For example, moderate rain at 10 mm/h does not appreciably affect the accuracy of LiDAR, but heavy rain exceeding 30 mm/h will see its effective measurable distance cut in half [45]. Yoneda et al. [12] highlighted another challenge of rain on LiDAR: vehicle-induced rain splashes from puddles in LiDAR data can mimic actual obstacles, complicating the interpretation of LiDAR observations, yet one must be able to distinguish between water splashes and actual obstacles in LiDAR data. Heavy rainfall can significantly degrade camera system performance by affecting image quality and obscuring object details. However, deep learning algorithms can effectively eliminate rain streaks from images, enhancing image quality in variable weather conditions [46]. Additionally, Yodeda et al. [12] suggested that positioning the camera within the wiper can mitigate this problem by removing raindrops from the camera’s field of view, enhancing visibility. Zang et al. [10] highlighted additional challenges, such as ice formation on rotating cameras and lens frosting under freezing conditions, suggesting self-heating cameras to maintain clear vision and prevent mechanical obstruction.

3.1.2. Snow and Hail Effects

Although rain is the principal factor in radar signal attenuation, it is crucial to recognize that snow and hail also present significant challenges to the operation of autonomous vehicles [47]. Vargas et al. [2] emphasized that all forms of precipitation, whether liquid or frozen, can impact the electromagnetic signals crucial for AV mechanisms. Battan et al. [48] and Lhermitte et al. [49] discussed radar attenuation due to hail, specifically addressing microwave and millimeter-wave radar (MWR), respectively. In snowy conditions, MWRs are impacted by attenuation and increased backscatter. Wallace et al. [42] highlighted the complexities in accurately assessing and predicting the impact of snow on radar signals. The diverse physical properties of snow, influenced by environmental and temporal factors, complicate the establishment of a consistent link between snow characteristics and radar performance. Wallace et al. [42] also noted that snow’s composition of crystallized water, which varies in shape, size, and free water content, adds to this complexity. This variability obstructs efforts to directly correlate radar signal attenuation with snow mass, free water content, and visibility, often resulting in inconsistent and unpredictable patterns.

Yoneda et al. [12] and Meydani et al. [50] reported that snowfall poses significant challenges for AVs regarding self-localization, object recognition, and path planning. Snow can obscure road markings and change the appearance of surrounding objects, making it difficult for AVs to match real-time observations with map data, affecting lane positioning and safe driving [12]. To enhance self-localization, methods like high-precision RTK-GNSS, reconstructing observation data, and using MWR are proposed. Although these offer varying degrees of accuracy, they help navigate snowy conditions.

Moreover, snowflakes are misinterpreted as obstacles by LiDAR sensors, which complicates object recognition and requires the use of noise removal techniques and machine learning in the same manner as rainfall measurements are reconstructed [51]. Another hurdle is path planning since snow can change the drivable space on roads. However, high-definition maps, which rely on clear conditions, are insufficient, and an adaptive approach that considers current weather and road conditions is required. In addition, snow or hail can block off portions of the camera’s field of view, causing blurred images that prevent effective image processing tasks, such as pattern recognition. Snowfall and hail also affect the quality of images captured by cameras and directly threaten the camera hardware, as highlighted by Zang et al. [10]. Hailstorms also present a risk of physical damage to camera lenses. In cold and snowy weather, cameras face additional challenges from low temperatures and optical and mechanical disruptions. In addition, cameras with no protective shielding are particularly vulnerable to damage from ice accumulation.

3.1.3. Fog Effects

Foggy conditions significantly impact the performance of LiDAR due to Mie scattering, as the operating wavelengths of LiDAR are generally smaller than the size of fog particles. When coupled with water absorption, this scattering has a pronounced effect on the Near-Infrared (NIR) spectral band [2], leading to a reduction in reflectance and a decrease in the measured distances from the LiDAR sensor. Research by Wojtanowski et al. [52] indicates that under foggy conditions, the effective visibility range of LiDAR at wavelengths of 905 nm and 1550 nm is considerably reduced, more so than under rainfall. The study further reveals that the 1.5 μm wavelength experiences greater water absorption, by an order of magnitude of two, compared to the 0.9 μm wavelength, making it more vulnerable in wet conditions. Reflectivity varies across surfaces; for instance, while some surfaces reflect more light at 1.5 μm when dry, they appear darker when wet. They also found that as fog density increases, causing visibility to drop from 500 m to 200 m, the ability of rangefinders to measure distances accurately diminishes correspondingly.

Hadj-bachir and Souza [45] have corroborated these findings, observing that the measurement distance of LiDAR is inversely related to fog density, with heavier fog leading to significantly reduced visibility and, consequently, measurement range. Kutila et al. [53] examined the performance of two LiDAR systems: the Ibeo Lux and Velodyne PUCK, which operate at the 905 nm wavelength. The study found that under foggy conditions, the detection efficiency of both LiDAR systems was halved. The potential shift to using the 1550 nm wavelength. Kutila et al. [54] also mentioned that testing under foggy conditions revealed adverse effects that led to a 25% reduction in sensor performance. Given these results, the researchers explored the feasibility of employing a 1550 nm wavelength by testing a “pre-prototype LiDAR” designed for this higher wavelength. The rationale for this shift was based on the fact that the 1550 nm wavelength meets eye safety standards, allowing for laser sources with power up to 20 times greater than those permissible at 905 nm. This increase in power could potentially enhance LiDAR functionality in low-visibility scenarios, such as heavy fog. However, further research is necessary to confirm the advantages of using the 1550 nm wavelength in adverse weather conditions.

Similarly, the range of camera measurements is compromised due to reduced visibility. In heavy fog, image recognition using visible light becomes challenging, even for human vision. This is illustrated in Figure 3, which shows how image quality and the visibility of illuminated objects, such as traffic lights, are notably degraded in fog. Fog also reduces the contrast of images, which is essential for pattern and edge recognition in image processing [10]. In an image, sharp edges are typically represented by a mix of low and high-frequency components, whereas smooth edges are depicted by low frequencies only. Fog affects these frequencies in an image: in a foggy scene, frequency components tend to be concentrated at zero frequency, leading to a loss of edge definition. Addressing these impediments, infrared cameras are proposed as a viable solution due to their operation on wavelengths that penetrate fog more effectively than visible light. Moreover, integrating deep neural networks has shown promise in processing LiDAR and camera data to mitigate performance deterioration under low visibility conditions by extracting salient information obscured by fog.

According to Yadav et al. [55], there are two methods to counteract fog in images: Fog correction which adjusts contrast levels, and fog removal, which estimates and algorithmically clears the fog from the image to improve visibility. These technological advancements allow for autonomous vehicles to operate safely and efficiently, regardless of bad weather. MWR, on the other hand, shows some degree of resilience when the foggy conditions, especially at shorter range (about 20 m) [53], warrant the consideration of the strengths of different sensors in different weather conditions [12]. Ultrasonic sensors are known to be less affected by scattering effects than other sensors. They employ sound waves and are sensitive to the air composition and temperature. Still, their performance is mainly insensitive to precipitation, as they are typically used in short-range detection applications.

3.1.4. Lightening Effects

Despite the potential impact of lightning on AVs, limited literature addresses this issue. Considering that AVs rely heavily on technologies driven by electricity and electromagnetism, it is important to consider various unpredictable conditions, including electrical interferences like lightning. When lightning strikes, it generates a substantial electromagnetic field, which has the potential to significantly affect electronic devices [56], including AVs [2]. This can disrupt the functioning of sensors, communication systems, and onboard computers, leading to temporary malfunction or, in extreme cases, permanent damage. Lightning can affect GPS systems and other navigation aids, leading to inaccuracies in positioning data. Moreover, communication systems used for Vehicle-to-Everything (V2X) communications might also be disrupted, affecting the AV’s ability to communicate with other vehicles and infrastructure. Vargas et al. [2] stated that lightning could have high direct effects on all AV sensors, high indirect effects on LiDAR and camera, and medium indirect effects on RADAR, GNSS, and ultrasonic sensors. The bright flash of lightning can also temporarily blind these sensors, causing a lapse in data collection or inaccuracies in object detection and ranging, which will be discussed widely in the light section.

3.1.5. Severe Light Effects

The overpowering brightness of sun glare is a major challenge to AV performance. MMW radar, ultrasonic sensors, and GNSS are less affected by sun glare, but cameras and LiDAR can suffer from performance degradation caused by intense light interference. Under sun glare, cameras, particularly, are susceptible to obscuring visual information needed by image recognition algorithms to accurately identify objects, resulting in either non-detection of objects such as pedestrians or false detection. Likewise, LiDAR sensors may have reduced accuracy because of light scattering and thus may be unable to detect objects at a distance. Figure 4 illustrates the adverse effects of sun glare on camera and LiDAR performance, with a Xenon light source generating 200 kilolux (klx) at 40 m distance. To mitigate these issues, the implementation of robust sensor fusion algorithms, advanced filtering techniques, and hardware improvements, such as high dynamic range cameras, are crucial. As demonstrated in Figure 5, the use of thermal cameras (top left inset), which detect heat instead of light, provides more reliable imaging under these conditions, showing that thermal cameras, along with thermal multiscale retinex and thermal bio-inspired retina techniques, significantly improve object recognition [13]. The recall rates for thermal bio-inspired retina, in particular, reached 67.1%, 45%, and 78.6% for a person, car 1, and car 2, respectively. In another study conducted by Yahiaoui et al. [57], the glare is detected by an image processing algorithm with several processing blocks, including color conversion, adaptive thresholding, geometric filters, and blob detection, and trained with a CNN network.

Sun glints, like sun glare, can sensor blind and lead to erroneous readings in cameras and LiDAR systems. Reflected light can provide false positives in object detection or can obliterate the existence of real obstacles. This is extremely difficult in environments with highly reflective surfaces like wet roads. This challenge is clearly illustrated by Vargas et al. [2], illustrating how a sun glint can appear on water surfaces and providing valuable insights into the real-world implications of this phenomenon for AVs.

At the same time, shadow and low light conditions add another layer of difficulty for AVs. However, these conditions reduce the sensor’s performance, but it can be compensated with different computer vision techniques. The performance of AVs is ensured by these techniques in various lighting environments and improves image quality and object detection capability. For example, the DriveRetinex-Net algorithm is a popular image enhancement method that increases visibility and sensor accuracy when limited light levels are available [58].

3.1.6. Dust and Sandstorm and Contamination Effects

Less frequent but more severe weather phenomena, including dust and sandstorm air, could also affect AV performance by obscuring sensors’ lenses, such as cameras and LiDAR, reducing visibility and accuracy. These conditions can become a big obstacle in regions such as the Middle East or desert areas. Sky Harbor International Airport reported [2] that dust storms can reduce visibility to 800 m. In these environments, cameras and LiDAR are the most disrupted sensors, with cameras being worse affected than LiDAR. This is a problem that Trierweiler et al. [59] have highlighted, as their tests of near-homogeneous dust particles on the surface of a LiDAR scanning system reduced the system’s maximum detection range by 75%. Mie scattering by dust storm particles with 2.5–10 microns diameters, similar to how fog or rain affects LiDAR [2], can also occur at LiDAR wavelengths of 905 and 1550 nm.

Similarly, smoke, which shares physical characteristics with dust storms, poses similar performance challenges for AVs. Visual cameras (0.4 to 0.7 μm) and near-infrared instruments like LiDAR’s (0.905 µm) face difficulties in dense smoke and cold conditions when visibility is very poor, as reported by Starr and Lattimer [60]. Additionally, as Kovalev et al. [61] investigated how lidar signals are affected by multiple scattering in dense smoke plumes, and the additional difficulties associated with sensor operation in such an environment were also explored.

Contamination, such as dirt, dust, and other opaque or semi-transparent materials, can significantly affect the performance of AV sensors, including backup cameras (as shown in Figure 6). Contaminants can obscure the sensors’ view, posing a severe challenge to the robustness and adaptability of autonomous driving systems. Uřičář et al. [62] developed a novel approach to address lens soiling in automated driving systems. Their study focused on wide-angle fisheye cameras used in parking and low-speed navigation, susceptible to soiling from mud, dust, water, and frost. They introduced a Generative Adversarial Network (GAN)-based algorithm capable of generating unseen patterns of soiled images and corresponding soiling masks, thus eliminating manual annotation. This method significantly improved the accuracy of soiling detection by up to 18%.

Trierweiler et al. [59] further advanced these efforts with an automatic control for windshield wipers and nozzles. It can discriminate between contaminants (liquid raindrops or solid dust) by observing total internal reflection and intensity distribution. This system differs from conventional rain sensors in deciding which pollutants to clean using cleaning nozzles. Thus, the problem of sensor contamination in autonomous driving is as important as other critical problems. Developing new methods to detect and combat these effects is necessary to make autonomous driving systems operate safely and efficiently.

In summary, adverse weather conditions can significantly impact the effectiveness of LiDAR, RADAR, and cameras unevenly. On the other hand, GNSS technology is relatively less sensitive and depends on high-frequency satellite signals that can pass through clouds and rain [2,12]. Nonetheless, adverse weather conditions cause minor signal disturbances. For example, GNSS modules placed indoors may have low signal visibility because of the oscillating windshield wiper blades, and those installed outdoors may suffer from signal fade due to raindrops on the exterior antenna [10]. Also, ionospheric scintillation, a space weather effect, may sometimes make tracking signals slightly difficult for a while [63].

Research on sensors for AVs in adverse weather conditions is crucial for the advancement of safe and reliable automated transportation systems. This will likely involve the evolution and improvement of multi-sensor fusion methodology, combining data from improved current models of LiDAR, RADAR, and cameras, along with complex environmental modeling. Robust sensor fusion techniques, which combine data from multiple sensors, such as radar, LiDAR, and cameras, have recently been developed to improve detection in adverse conditions. For example, fusion architectures based on neural networks have been demonstrated to improve reliability by compensating for the weaknesses of individual sensors, especially in the presence of snow and fog. Further, adaptive domain incremental learning algorithms have shown improvements by dynamically handling sensor data distortions to maintain consistent object detection performance across different weather domains. Also, artificial intelligence is essential; the AVs must learn to make better choices by using better algorithms to analyze the weather and how it becomes dynamic.

Another interesting application is in selecting materials and advanced technologies that do not allow the formation of obstacles to sensors, such as hydrophobic coatings and self-cleaning. Sandstorms present special challenges because of their effects on sensor hardware. LiDAR beams are scattered by dust particles, which obscure camera lenses, significantly degrading performance. These limitations are being addressed through innovations such as self-cleaning sensor enclosures and hydrophobic coatings. There will also be a need for cooperation between meteorologists, sensor technologists, and artificial intelligence engineers to generate models that drive AVs and assist them in overcoming major precipitation challenges.

Therefore, solving these challenges will require strong sensor hardware, sophisticated algorithms, and interdisciplinary cooperation. Future work has to focus on real-world testing of resilient autonomous vehicle systems, and that testing must be performed in extreme weather conditions. Table 3 demonstrates a summary of some research on this area.

Table 3. Research projects focused on advancing AV performance in adverse weather conditions.

Category	Sensor	Adverse Weather	References	Contribution	Challenges
CNNs and Variants	Camera	Rain	[64,65,66,67,68,69,70,71,72,73]	▪ Robust feature extraction ▪ Adaptability with variants ▪ Transfer learning ▪ Integration with other sensor data ▪ End-to-end learning ▪ Vision augmentation ▪ Simplifies high-dimensional data.	▪ Susceptibility to noise ▪ Computational intensity ▪ Overfitting risk ▪ Challenges of up-sampling and down-sampling ▪ Generalization issues ▪ Some algorithms use restricted adverse weather model assumptions. ▪ Mostly have been applied on a synthetic dataset. ▪ In some cases, small datasets have been used. ▪ In some cases, radar Doppler information was not incorporated. ▪ Challenges of large, sparse grid maps in system complexity ▪ Information loss
		Snow	[74]
		Rain, Fog	[75,76]
		Haze and Fog	[9,77,78,79,80,81,82]
	LiDAR	Rain	[8]
		All Weather	[83]
RNNs and Variants	Camera	Rain	[84,85,86]	▪ Temporal dependency handling ▪ Vision augmentation ▪ Contextual information processing ▪ Flexibility in input Sequence length ▪ Real-time decision-making. ▪ Handling sensor fusion data. ▪ Simplifies high-dimensional data. ▪ Continuous learning and adaptation ▪ Predictive maintenance and anomaly detection	▪ Computational demands ▪ Training challenges ▪ Overfitting ▪ Sequential data dependency ▪ Mostly have been applied on a synthetic dataset. ▪ In some cases, small datasets have been used. ▪ Challenges of large, sparse grid maps in system complexity ▪ Information loss
GANs and Related Techniques	Camera	Rain	[87,88,89,90,91,92,93,94]	▪ Data augmentation ▪ Image enhancement and restoration ▪ Simulating sensor data for radar or LiDAR ▪ Improving perception and Decision-making ▪ Transfer learning ▪ Vision augmentation	▪ Training complexity and stability ▪ Resource intensity ▪ Realism and trustworthiness of synthetic data ▪ Overfitting to synthetic features ▪ Ethical and safety considerations: ensuring accuracy and reliability in real-world applications. ▪ Mostly have been applied on synthetic datasets. ▪ In some cases, small dataset has been used
		Snow	[95,96]
		Haze and Fog	[97,98,99,100,101,102,103,104,105]
		Soil	[62]
Fusion and Multi-Modal Networks	Camera	Rain	[70,93,106,107,108,109,110,111]	▪ Enhanced perception accuracy ▪ Redundancy and reliability ▪ Improved object detection and classification ▪ Greater situational awareness ▪ Enhanced adaptability ▪ Vision augmentation	▪ Algorithmic challenges ▪ Latency issues ▪ Increased computational load. ▪ Complexity in integration ▪ Needs data management and storage solutions. ▪ Mostly have been applied on a synthetic dataset. ▪ In some cases, small dataset has been used
		Haze	[112,113,114,115,116,117,118]
Domain Adaptation and Unsupervised/Semi-Supervised Learning	LiDAR	Rain	[119,120]	▪ Reducing the need for labeled data. ▪ Solving the data absence problem ▪ Improving generalization ▪ Continuous learning ▪ Cost-efficiency ▪ Enhancing robustness ▪ Vision augmentation	▪ Complexity in implementation ▪ Potential for reduced accuracy ▪ Dependency on the source domain ▪ Risk of model drift ▪ Evaluation challenges ▪ Mostly, have been applied on synthetic datasets. ▪ In some cases, small dataset has been used
	Camera	Rain and Haze	[121]
		Haze	[122,123,124,125]
		Rain and Snow	[126]
Specialized Segmentation and Detection Networks	Camera	Rain	[74,127,128,129,130,131,132,133,134,135,136]	▪ Customizable for specific scenarios ▪ Efficient processing ▪ Robust to environmental variabilities ▪ Vision augmentation ▪ Enhanced detection accuracy	▪ Limited flexibility ▪ High development costs ▪ Information loss ▪ Resolution degradation ▪ Overfitting risk ▪ Maintenance and updates ▪ Integration complexity ▪ Mostly have been applied on a synthetic dataset. ▪ In some cases, small dataset has been used
		Snow	[137,138]
		Haze and Fog	[108,139,140,141,142,143,144]
	LiDAR	Rain and Snow	[145]
Simulation and Testing	LiDAR	Rain, Fog	[45,146]	▪ Controlled testing environment ▪ Cost effective. ▪ Enhanced data collection and analysis ▪ Rapid iterative testing	▪ Realism gap issues ▪ Model accuracy complexity ▪ Sensor simulation challenges ▪ Scenario design dependency ▪ Complacency risk concerns
		Snow	[147]
Sensor Fusion	LiDAR + Camera	Rain, Haze, and Fog	[148]	▪ Comprehensive data analysis ▪ Adaptability to environmental changes ▪ Robustness to sensor failures ▪ Enhanced object detection and classification ▪ Increased reliability and redundancy ▪ Increased operational range. ▪ Enhanced decision-making ▪ Compensate for the weaknesses of individual sensors	▪ High computational demands ▪ Increased system cost ▪ Algorithmic challenges ▪ Latency issues ▪ Complex integration of diverse sensor data ▪ Advanced algorithms needed for data fusion
		Rainy and Snowy	[149]
		Fog	[150]
	RADAR + Camera	Snow, Fog, Rain	[151]
	LiDAR + RADAR	Haze and Fog	[152]
		Rain, Smoke	[153]
Hardware Enhancements	LiDAR	Snow	[154]	▪ Improved sensor capabilities ▪ Enhanced data processing power ▪ Increased robustness and durability ▪ Reduced sensor failures ▪ Better integration of multi-modal systems	▪ Higher costs ▪ Increased complexity ▪ Greater power consumption ▪ Integration challenges ▪ Weight and space issues

Table 3. Research projects focused on advancing AV performance in adverse weather conditions.

Category	Sensor	Adverse Weather	References	Contribution	Challenges
CNNs and Variants	Camera	Rain	[64,65,66,67,68,69,70,71,72,73]	▪ Robust feature extraction ▪ Adaptability with variants ▪ Transfer learning ▪ Integration with other sensor data ▪ End-to-end learning ▪ Vision augmentation ▪ Simplifies high-dimensional data.	▪ Susceptibility to noise ▪ Computational intensity ▪ Overfitting risk ▪ Challenges of up-sampling and down-sampling ▪ Generalization issues ▪ Some algorithms use restricted adverse weather model assumptions. ▪ Mostly have been applied on a synthetic dataset. ▪ In some cases, small datasets have been used. ▪ In some cases, radar Doppler information was not incorporated. ▪ Challenges of large, sparse grid maps in system complexity ▪ Information loss
		Snow	[74]
		Rain, Fog	[75,76]
		Haze and Fog	[9,77,78,79,80,81,82]
	LiDAR	Rain	[8]
		All Weather	[83]
RNNs and Variants	Camera	Rain	[84,85,86]	▪ Temporal dependency handling ▪ Vision augmentation ▪ Contextual information processing ▪ Flexibility in input Sequence length ▪ Real-time decision-making. ▪ Handling sensor fusion data. ▪ Simplifies high-dimensional data. ▪ Continuous learning and adaptation ▪ Predictive maintenance and anomaly detection	▪ Computational demands ▪ Training challenges ▪ Overfitting ▪ Sequential data dependency ▪ Mostly have been applied on a synthetic dataset. ▪ In some cases, small datasets have been used. ▪ Challenges of large, sparse grid maps in system complexity ▪ Information loss
GANs and Related Techniques	Camera	Rain	[87,88,89,90,91,92,93,94]	▪ Data augmentation ▪ Image enhancement and restoration ▪ Simulating sensor data for radar or LiDAR ▪ Improving perception and Decision-making ▪ Transfer learning ▪ Vision augmentation	▪ Training complexity and stability ▪ Resource intensity ▪ Realism and trustworthiness of synthetic data ▪ Overfitting to synthetic features ▪ Ethical and safety considerations: ensuring accuracy and reliability in real-world applications. ▪ Mostly have been applied on synthetic datasets. ▪ In some cases, small dataset has been used
		Snow	[95,96]
		Haze and Fog	[97,98,99,100,101,102,103,104,105]
		Soil	[62]
Fusion and Multi-Modal Networks	Camera	Rain	[70,93,106,107,108,109,110,111]	▪ Enhanced perception accuracy ▪ Redundancy and reliability ▪ Improved object detection and classification ▪ Greater situational awareness ▪ Enhanced adaptability ▪ Vision augmentation	▪ Algorithmic challenges ▪ Latency issues ▪ Increased computational load. ▪ Complexity in integration ▪ Needs data management and storage solutions. ▪ Mostly have been applied on a synthetic dataset. ▪ In some cases, small dataset has been used
		Haze	[112,113,114,115,116,117,118]
Domain Adaptation and Unsupervised/Semi-Supervised Learning	LiDAR	Rain	[119,120]	▪ Reducing the need for labeled data. ▪ Solving the data absence problem ▪ Improving generalization ▪ Continuous learning ▪ Cost-efficiency ▪ Enhancing robustness ▪ Vision augmentation	▪ Complexity in implementation ▪ Potential for reduced accuracy ▪ Dependency on the source domain ▪ Risk of model drift ▪ Evaluation challenges ▪ Mostly, have been applied on synthetic datasets. ▪ In some cases, small dataset has been used
	Camera	Rain and Haze	[121]
		Haze	[122,123,124,125]
		Rain and Snow	[126]
Specialized Segmentation and Detection Networks	Camera	Rain	[74,127,128,129,130,131,132,133,134,135,136]	▪ Customizable for specific scenarios ▪ Efficient processing ▪ Robust to environmental variabilities ▪ Vision augmentation ▪ Enhanced detection accuracy	▪ Limited flexibility ▪ High development costs ▪ Information loss ▪ Resolution degradation ▪ Overfitting risk ▪ Maintenance and updates ▪ Integration complexity ▪ Mostly have been applied on a synthetic dataset. ▪ In some cases, small dataset has been used
		Snow	[137,138]
		Haze and Fog	[108,139,140,141,142,143,144]
	LiDAR	Rain and Snow	[145]
Simulation and Testing	LiDAR	Rain, Fog	[45,146]	▪ Controlled testing environment ▪ Cost effective. ▪ Enhanced data collection and analysis ▪ Rapid iterative testing	▪ Realism gap issues ▪ Model accuracy complexity ▪ Sensor simulation challenges ▪ Scenario design dependency ▪ Complacency risk concerns
		Snow	[147]
Sensor Fusion	LiDAR + Camera	Rain, Haze, and Fog	[148]	▪ Comprehensive data analysis ▪ Adaptability to environmental changes ▪ Robustness to sensor failures ▪ Enhanced object detection and classification ▪ Increased reliability and redundancy ▪ Increased operational range. ▪ Enhanced decision-making ▪ Compensate for the weaknesses of individual sensors	▪ High computational demands ▪ Increased system cost ▪ Algorithmic challenges ▪ Latency issues ▪ Complex integration of diverse sensor data ▪ Advanced algorithms needed for data fusion
		Rainy and Snowy	[149]
		Fog	[150]
	RADAR + Camera	Snow, Fog, Rain	[151]
	LiDAR + RADAR	Haze and Fog	[152]
		Rain, Smoke	[153]
Hardware Enhancements	LiDAR	Snow	[154]	▪ Improved sensor capabilities ▪ Enhanced data processing power ▪ Increased robustness and durability ▪ Reduced sensor failures ▪ Better integration of multi-modal systems	▪ Higher costs ▪ Increased complexity ▪ Greater power consumption ▪ Integration challenges ▪ Weight and space issues

3.2. Complex Environment Conditions

Navigating through complex environments poses substantial challenges to the performance of AVs, especially in terms of GNSS reliability. Dense urban areas with high buildings or canyons and harsh natural environments like mountains or forests, tunnels, and airports can hinder AVs’ ability to receive the signals critical for accurate navigation [155,156,157]. Figure 7 illustrates a typical situation within an urban canyon. The direct line-of-sight (LOS) signals from many, sometimes most, satellites are blocked, attenuated, or reflected by existing obstacles [158], leading to phenomena such as cycle slips [159] and multipath or non-LOS (NLOS) signals [160,161,162]. Signal attenuation results in noisy data, while multipath and NLOS effects caused by signal reflections off various obstacles introduce outliers in the measurements [163]. These challenges substantially impact the accuracy of AVs positioning. Figure 7 shows how tall buildings can block signals from satellites at low to medium elevations, highlighting the navigational challenges encountered in these settings.

As the deployment of additional satellites from other GNSS constellations increases, the availability of direct LOS signals for users equipped to handle complex environments is improving [164]. Simultaneously, researchers explore innovative solutions to enhance signal reception and processing in challenging areas. These efforts include integrating GNSS with other sensors such as Inertial Navigation System (INS) technologies (GNSS/INS), integration of Dead Reckoning (DR) with GNSS/INS data [155,165,166,167], advanced algorithms for detecting and mitigating multipath and NLOS signals [161,168], developing more sophisticated models for urban signal propagation [164], developing advanced algorithms for improving the resilience and performance of navigation systems [155,169,170,171,172,173,174]. Combining these technological advancements and increased satellite coverage could substantially mitigate the navigation challenges currently faced by AVs, paving the way for more reliable and accurate GNSS-based positioning even in the most demanding environments. Table 4 summarizes the recent research studies implemented to improve the performance of AV sensors in complex environments.

Recent research on AVs in complex environments has focused on various aspects of their development and implementation. This collection of papers explores different aspects of autonomous navigation and control for AVs in complex environments.

Table 4. Research projects focused on advancing AV performance in complex environments.

Category	References	Contribution	Challenges
Deep Learning Models	[6,175]	▪ Simplifies motion planning tasks. ▪ Enables real-time decision-making. ▪ Improves decision-making using local or networked data. ▪ Ensures accuracy in GNSS-denied environments.	▪ Scalability in complex urban and dynamic environments ▪ Sensitive to noise and occlusions. ▪ Risk of overfitting. ▪ Struggles with rapidly changing conditions.
Sensory Data Integration	[7,176,177]	▪ Enhances situational awareness and decision-making with sensor data fusion.	▪ Managing large, real-time sensor data. ▪ Challenges in unpredictable or harsh environments.
Simulation and Testing	[178,179]	▪ Uses simulations to evaluate AV performance in complex scenarios. ▪ Allows rapid testing without real-world risks.	▪ Issues with simulation realism and trustworthiness. ▪ Limited power and problems in low-light and cluttered environments.
Autonomous Navigation	[180,181,182]	▪ Provides navigation techniques for urban and 3D environments. ▪ Manages dynamic obstacles and complex path planning in real time.	▪ Difficulties in pathfinding and obstacle avoidance in 3D environments. ▪ Needs sophisticated algorithms for real-time changes.
Reinforcement Learning	[183,184,185]	▪ Develop algorithms for decentralized navigation. ▪ Adapts to navigation challenges in complex environments.	▪ Ensuring model robustness in changing environments. ▪ Balancing exploration, exploitation, and safety.
Collaborative Systems	[186,187]	▪ Facilitates cooperative control for tasks in complex settings. ▪ Enhances task management through vehicle collaboration.	▪ Vehicle integration and coordination challenges.
Sensor Fusion	[188,189]	▪ Improves state estimation and situational awareness with sensor fusion. ▪ Increases positioning accuracy in GNSS-limited environments.	▪ Sensor accuracy limitations and environmental adaptation. ▪ Signal interference, especially in NLOS conditions.

Table 4. Research projects focused on advancing AV performance in complex environments.

Category	References	Contribution	Challenges
Deep Learning Models	[6,175]	▪ Simplifies motion planning tasks. ▪ Enables real-time decision-making. ▪ Improves decision-making using local or networked data. ▪ Ensures accuracy in GNSS-denied environments.	▪ Scalability in complex urban and dynamic environments ▪ Sensitive to noise and occlusions. ▪ Risk of overfitting. ▪ Struggles with rapidly changing conditions.
Sensory Data Integration	[7,176,177]	▪ Enhances situational awareness and decision-making with sensor data fusion.	▪ Managing large, real-time sensor data. ▪ Challenges in unpredictable or harsh environments.
Simulation and Testing	[178,179]	▪ Uses simulations to evaluate AV performance in complex scenarios. ▪ Allows rapid testing without real-world risks.	▪ Issues with simulation realism and trustworthiness. ▪ Limited power and problems in low-light and cluttered environments.
Autonomous Navigation	[180,181,182]	▪ Provides navigation techniques for urban and 3D environments. ▪ Manages dynamic obstacles and complex path planning in real time.	▪ Difficulties in pathfinding and obstacle avoidance in 3D environments. ▪ Needs sophisticated algorithms for real-time changes.
Reinforcement Learning	[183,184,185]	▪ Develop algorithms for decentralized navigation. ▪ Adapts to navigation challenges in complex environments.	▪ Ensuring model robustness in changing environments. ▪ Balancing exploration, exploitation, and safety.
Collaborative Systems	[186,187]	▪ Facilitates cooperative control for tasks in complex settings. ▪ Enhances task management through vehicle collaboration.	▪ Vehicle integration and coordination challenges.
Sensor Fusion	[188,189]	▪ Improves state estimation and situational awareness with sensor fusion. ▪ Increases positioning accuracy in GNSS-limited environments.	▪ Sensor accuracy limitations and environmental adaptation. ▪ Signal interference, especially in NLOS conditions.

3.3. Road Infrastructure Conditions

Road signs are critical for the localization and navigation of AVs, and they are heavily dependent on visible curves, road edges, speed limits, etc., for their practical interpretation of traffic rules [190,191]. Sensors, cameras, artificial intelligence, etc., are used to identify elements by detecting markings, colors, shapes, messages, etc. [192]. Still, the existing road infrastructure of markings and signage, which is on the road must be sufficiently adapted or maintained to accommodate AV technology with the stringent requirements. For example, U.S. Road network challenges such as varied traffic sign designs, faded road markings, and using paints with varying reflectivity and lack of necessary markings pose discontinuities. Liu et al. [193] also showed other obstacles that could make the use of machine vision systems problematic, such as the low maintenance priority of road signs in the UK, which means that they can become unreadable because of dirt; the confusion from EU nonstandard road signs; and temporary changes to road layouts caused by incidents and road works. Also, Bruno et al. [192] pointed out the false positives and negatives in signal detection by infrastructure improvement and standardization of AV navigation systems so that efficiency and safety can be improved. Vital signs or markings can be hidden behind these obstacles, such as vehicles parked along roadsides, vegetation, and existing roadside infrastructure [194]. Inadequate infrastructure devastates AV sensor performance, thus localizing, navigating, and making safe decisions. From faded road markings to inconsistent signage, camera-based systems misread lane boundaries or do not follow key traffic rules. Accurate environmental modeling can also be hampered by worn or obscured markings, as with LiDAR and radar systems. Nevertheless, these issues are especially critical in adverse weather conditions or nighttime operations when the lack of high contrast markings degrades detection accuracy. There are no consistent and visible markings to help AVs stay in the lane or respond appropriately to dynamic traffic situations. The phenomenon of “ghost markings”, resulting from hydro-plastic, adds another layer of complexity, potentially confusing AVs [194]. Road design and maintenance inconsistencies like abrupt transitions from one pavement type to another or poorly maintained intersections add other complexities. Furthermore, road construction or detours add temporary disruptions to these problems, requiring that AVs use prediction algorithms that may not fully capture the variability in infrastructure. These challenges highlight the need for standardized road construction practices and maintenance to ensure reliable AV operation. Furthermore, road drainage systems can significantly impact AVs’ performance, especially in areas susceptible to intense rainfall. The presence of water on the road and surfaces muddied by runoff due to inadequate drainage can hinder the ability of AVs to discern lane markings and road edges [191]. The problem becomes even more pronounced at night [195]. Rural and remote areas exacerbate these challenges, often lacking the infrastructure and communication networks vital for AVs’ optimal functionality.

These challenges clearly indicate the importance of standardization and upgrading road infrastructure (road markings, traffic signs, drainage, roadside, etc.) to allow the safe operation of different road conditions [193,195]. In recent years, researchers have been increasingly interested in finding out the best requirements and conditions for road markings for AVs’ safe navigation [196]. To address these needs, innovative solutions, such as the smart signs and markings developed by 3M, a US leader in traffic signage and road marking production, are being introduced to fill the gap with advanced materials and technology meeting the challenging needs of AV navigation [194]. Nevertheless, several studies have recommended that infrastructure be maintained as it is in the early stages of AV integration [193]. Real-time detection of lane boundaries and vehicle guiding has been studied in several studies to develop algorithms that can do this [7,197,198,199]. Research studies also highlighted the need for physical and digital infrastructure refreshes to facilitate safe and efficient AV operation with few deficiencies.

Both physical enhancements in the structure and technological improvements are needed to overcome the deficiencies of the existing systems. These challenges require advances in infrastructure design and vehicle control technologies. For instance, curve passing control models that combine camera-based lane detection with vehicle dynamics are shown to improve navigation accuracy by adjusting speed and steering based on real time lane curvature data. These methods enhance vehicle stability and passenger comfort in situations where high speed cornering or unclear lane markings are present. In addition, the integration of adaptive control strategies, including Model Predictive Control based ones, guarantees precise trajectory tracking and reliable vehicle performance under changing road and environmental conditions [200].

This collection examines different aspects of autonomous navigation and control for AVs in roadway infrastructure. However, it is notable that most of this research has investigated the problem through the prism of hardware and software capabilities, particularly image capture devices and detection algorithms.

Table 5. Research projects addressing challenges and advancements in road infrastructure to support AVs.

Category	References	Contribution	Challenges
Road Infrastructure Upgrades	[193,201,202]	▪ Created long-term plans for upgrading AV infrastructure. ▪ Established knowledge bases for AV infrastructure needs.	▪ Major investments and upgrades needed. ▪ Collaboration between agencies and sectors is needed for standardization.
Digital Infrastructure and V2X Communication	[202,203,204,205]	▪ Supports V2V/V2I communication to improve safety and efficiency. ▪ Enables real-time data for better traffic flow and hazard detection. ▪ Controls traffic signals dynamically to reduce congestion. ▪ Enhances AV navigation by integrating real-time infrastructure data.	▪ Challenges in ensuring fast, reliable, secure V2X communication. ▪ High costs and complexity in scaling V2X across large networks.
Management and Planning Models	[201,206,207,208]	▪ Developed models for infrastructure upgrades, focusing on reliability-based planning. ▪ Proposed strategies to integrate AVs into future infrastructure.	▪ Infrastructure management needs to be updated for new technologies. ▪ Advanced tools required to manage future infrastructure complexity.
Human-Centered and Cooperative Design	[205,209,210]	▪ Designed frameworks for smooth AV and HDV interaction. ▪ Ensured infrastructure works for both human drivers and AVs.	▪ Risk of miscommunication between AVs and HDVs needs careful design. ▪ Infrastructure must be optimized for both human drivers and AVs.
Simulation and Testing Environments	[147,205,211,212]	▪ Developed simulation frameworks to test and validate infrastructure support for AVs. ▪ Simulation improves AV testing in traffic conditions.	▪ Difficulties in creating realistic simulations that reflect real-world conditions. ▪ Simulation models must capture both AV and human driver behavior accurately.
Sensor Integration and Optimization	[205,208,213,214]	▪ Proposed frameworks for sensor integration in infrastructure. ▪ Emphasized real-time traffic data for safer driving.	▪ Ongoing need to optimize sensor integration and data processing. ▪ Connectivity gaps can impact real-time traffic monitoring performance.
Infrastructure Impact Studies	[11,206,212,215]	▪ Analyzed AV impact on road infrastructure for planning. ▪ Explored long-term effects of AVs on-road use and maintenance.	▪ Mixed traffic during the transition period complicates infrastructure planning. ▪ Infrastructure must stay adaptable to evolving AV technology, requiring frequent updates.

also shows the most recent studies highlighting how infrastructure is a critical enabler for AVs.

Based on these different methods for improving the performance of AVs in critical situations, the effectiveness of these systems is strongly tied to the robustness of the algorithms that power their perception, decision-making, and control systems. A large portion of these algorithms, in particular those based on deep learning and machine learning, are data driven. The training on large, diverse datasets is extensive, and these methods are not yet accurate enough to detect and interpret real-world conditions, like objects in urban environments, road structures, weather conditions, and dynamic obstacles. These algorithms are effective partly because of how they are designed but also because the data we use to train them is of high quality and diverse. Without adequate datasets, models can fail to generalize to novel, complex scenarios and produce disastrous failures in critical circumstances.

4. Datasets Availability

For improving algorithms in AVs in critical situations, several specialized datasets are widely used to train and evaluate perception systems and algorithms. The availability of datasets is important for the development and cross-evaluation of methods. Creating large-scale datasets for training algorithms from scratch is not easy; therefore, pre-existing publicly available datasets are very important because they accelerate the development of algorithms, and researchers are not required to build their datasets themselves. In this section, we survey the available datasets and

Table 6. List of datasets used for AV research and development.

Dataset	Type of Data	Reference
Rain12600	Synthetic	[216]
Rain100L	Synthetic	[65,66,67,68,71,72,73,84,90,91,127,128,129,132,134,135]
Rain200H	Synthetic	[64,71,127,133]
Rain800	Real	[76,89,90,91,109,127,129,134]
Rain12	Synthetic	[64,66,68,87,109,134]
Test100	Synthetic	[68,70,73,89,132]
Test1200	Synthetic	[70,73,110]
Test2800	Synthetic	[70]
RainTrainH	Synthetic	[217]
RainTrainL	Synthetic	[217]
Rain100H	Synthetic	[65,67,70,71,72,73,84,91,93,107,109,111,127,129,132,134,135,136]
KITTI	Real	[76,107,150,218,219,220]
Raindrop	Real	[108,133]
Cityscapes	Real	[62,70,107,121,128]
RID	Real	[70,109]
RIS	Real	[70,109]
Rain12000	Synthetic	[85]
Rain1400	Synthetic	[64,71,72,107,109,131,135]
DAWN/Rainy	Real	[221]
NTURain	Synthetic	[86,109]
SPA-Data	Real	[90,127]
RESIDE	Synthetic and Real	[76,77,78,97,139,144]
I-HAZE	Synthetic	[77,222]
O-HAZE	Synthetic	[77]
DENSE-HAZE	Synthetic	[77]
NH-HAZE	Synthetic	[77,144]
HazeRD	Synthetic	[78,144]
SOTS	Synthetic	[78,108]
4KID	Synthetic	[223]
BeDDE	Real	[224]
REVIDE	Synthetic	[225]
Snow-100K	Synthetic and Real	[95]
SITD	Synthetic	[226]
CADC	Real	[227]
WADS	Real	[228]
CSD	Synthetic	[137]
SRRS	Synthetic and Real	[138]
AVPolicy	Synthetic	[6]

summarizes the proposed datasets. These datasets can be broadly categorized into synthetic and real data, offering unique advantages for improving model performance. Synthetic datasets simulate adverse weather conditions such as rain and haze, allowing for controlled, large-scale experimentation where ground truth is known. On the other hand, real-world datasets capture actual urban environments and natural weather effects, providing valuable data for testing the generalization of algorithms in realistic scenarios. Some datasets, like SPA-Data and BeDDE, also focus specifically on urban driving conditions, while others offer a combination of synthetic and real data to improve robustness across weather conditions.

5. Future Research Directions

Based on the critical situations described in this review and the various sensors and methods’ strengths and limitations to improve AV performance, we observe that while many algorithms intended for these scenarios look promising, most have not been validated in real-world conditions. Future research must focus on testing these algorithms in a practical environment, specifically through unsupervised learning. Models can learn directly from large amounts of real-world data without manual labeling using these methods. This is important for AVs because real-world environments are highly dynamic and unpredictable, so it is not feasible to manually label all possible scenarios. By unsupervised learning, the system learns to find patterns, adapt to new situations, and continuously improve its performance in real time, narrowing the gap between the data from simulations and real conditions and reducing the time and cost of labeled datasets. Such unified models would also significantly improve AV performance by enabling them to cope with multiple critical conditions. Furthermore, applying these methodologies to more challenging driving scenarios, like highway merging and dense urban areas, is necessary to advance this work further.

Currently, most available datasets are synthetic. While synthetic datasets have been invaluable for early development, incorporating real-world data from ego vehicle perception systems is a key step toward improving the reliability and robustness of AV algorithms in real driving conditions. Similarly, simulation studies, another dominant approach in AV research, need rigorous real-world validation to ensure their effectiveness. Bridging the gap between simulated models and real-world data is essential for the successful deployment of AVs in unpredictable, dynamic environments.

Autonomous driving research relies heavily on simulated data that allows controlled experiments with scalability. Real-world data are, however, still required to understand the full spectrum of naturalistic driving scenarios. Research also shows that challenges in real-world data collection, including high costs, privacy concerns, and unpredictability of environmental conditions, call for the complementary use of simulation tools [13]. These challenges are mitigated by annotated datasets and advanced simulators, which enable more complete testing and modeling of autonomous vehicle systems. Due to the complexity of autonomous vehicle operations in diverse and adverse conditions, it is critical to integrate simulation with real-world data [13].

To overcome these challenges, a hybrid approach that integrates simulation with real-world testing is necessary. Developing modular sensor platforms capable of transitioning seamlessly between simulated and real-world environments is also critical. A collaborative effort among researchers, industry, and municipalities will facilitate these pilot programs and accelerate the transition from research to real-world applications.

6. Discussion, Conclusions, and Future Work

6.1. Discussion

AVs have extensive possibilities to enhance road safety, traffic flow, and energy utilization and reduce fatal crashes. These benefits could lead our traffic operation system to fully automated Level 5 traffic. At the same time, we understand that realizing this vision and acquiring public trust on a broader scale could be possible only when expanding our research in sensor algorithms, perception, and control mechanisms have been made.

Our review shows that AVs still face challenges in adverse weather, complex environments, and roadway infrastructure and a lack of public trust and safety concerns remain a major obstacle to broader public acceptance. However, transparency and trust levels do not improve while technology progresses continuously. Most people still do not fully trust AVs to take care of several things they value most, such as having their children travel with AVs. Such a feeling shows the challenge of having practical test data to yield a comprehensive safety concern. Even market leaders like Waymo and Cruise continue to experience such delays in AVs’ mass production; this indicates that even the leading companies in the industry have not addressed these trust-related problems fully and are not confident to provide their technologies to the public yet.

This review suggests that to obtain full acceptance of AVs in society, the cooperation of stakeholders in the automotive industry, policymakers, and researchers is imperative to close this trust gap. Policymakers should promote AVs in the same manner as solar panels and electric cars as a solution that raises traffic safety and efficiency through state incentives such as tax exemptions for their usage. In addition, we believe that open data sharing with the research community would be necessary, especially data from automation levels 3, 4, and 5. Finally, from the perspective of adverse weather to complex environments and roadway infrastructures, we anticipate that testing under various conditions would be essential. We believe this collaboration would invigorate research into developing more accurate algorithms, improving current technologies, and enhancing public understanding and acceptance of AVs.

6.2. Conclusions and Future Work

This review assessed the performance of four types of sensors, LiDAR, RADAR, Camera, and GNSS, under three critical situations: adverse weather, complex environments, and road infrastructure.

6.2.1. Adverse Weather Summary

Adverse weather conditions significantly affect the performance and reliability of AVs, particularly impacting key sensors like LiDAR, RADAR, and cameras. Rain causes radar signal attenuation and backscatter, reducing detection capabilities, while snow and hail further complicate radar performance by introducing variability in signal attenuation. These conditions also obscure road markings and alter object appearances, complicating self-localization and object recognition. Fog reduces LiDAR and camera accuracy, but advancements like the use of higher wavelength LiDAR, infrared cameras, and deep learning techniques show promise in mitigating these effects. Sun glare and dust storms similarly degrade sensor performance, though solutions like thermal cameras and sensor fusion have been proposed to enhance visibility. Despite these challenges, GNSS remains relatively stable but may experience minor disruptions in extreme weather.

Gap Identified: Most research has focused on camera-based solutions, with less attention given to enhancing LiDAR and RADAR technologies for adverse weather conditions. Snow and other extreme weather events, such as sandstorms, strong light, and sensor contamination, remain underexplored in terms of their impact on AV sensors.

Future Work: Future research should focus on improving the performance of sensors in snow, sandstorms, and other challenging conditions. This includes the development of robust sensor fusion algorithms that can integrate data from multiple sources to enhance overall situational awareness and decision-making. Expanding datasets that include more diverse weather conditions will also help improve the generalizability of existing models.

6.2.2. Complex Environments Summary

Navigating complex environments poses significant challenges for AVs, particularly with GNSS reliability. In urban canyons and tunnels, satellite signals are often blocked, reflected, or attenuated, leading to positioning errors. To address this, researchers are integrating GNSS with Inertial Navigation Systems (INS) and Dead Reckoning (DR) and developing algorithms to mitigate multipath and non-line-of-sight (NLOS) signal issues. Increased satellite coverage and improved signal processing techniques are also helping to enhance positioning accuracy in these demanding environments, making navigation more reliable for AVs.

Gap Identified: Despite progress, the integration of GNSS with INS and other systems is still not robust enough to handle extreme environmental conditions consistently. Moreover, the complexity of signal propagation in urban environments remains an open challenge, requiring more advanced models and algorithms.

Future Work: Research should focus on developing more sophisticated algorithms for detecting and mitigating multipath and NLOS signals. Additionally, improving GNSS/INS and Dead Reckoning (DR) integration will be critical for ensuring reliable positioning in dense urban areas, mountainous regions, and other complex environments. Future studies should also aim to improve signal processing in obstructed environments, with an emphasis on real-time adaptation to signal fluctuations.

6.2.3. Road Infrastructure Summary

AVs rely heavily on road features like curves, edges, signage, and markings for navigation, but current infrastructure often falls short of meeting their requirements. Issues such as faded markings, inconsistent sign designs, and obstacles like parked vehicles or vegetation can impair AVs’ ability to navigate safely. In areas with heavy rainfall or poor drainage, water and mud can obscure lane markings, particularly at night, while rural and remote areas often lack the necessary infrastructure and communication networks. These challenges highlight the need for standardized and upgraded road infrastructure, including advanced materials and smart signs. Efforts are also underway to develop real-time detection algorithms for lane boundaries and vehicle guidance. While some researchers advocate for preserving existing infrastructure during early AV integration, it is clear that both physical and digital upgrades are essential for safe and efficient AV operations.

Gap Identified: Current road infrastructure does not consistently meet the needs of AV systems. Inadequate standardization and maintenance, especially in rural or underdeveloped areas, create gaps that reduce the effectiveness of machine-vision systems. Additionally, temporary road alterations due to construction or roadworks introduce further complexities.

Future Work: There is a critical need for the standardization and upgrade of road infrastructure, including consistently using high-visibility road markings and signs. Future work should also explore innovative solutions like smart signs and advanced materials for road markings, which can improve the interpretability of infrastructure for AV sensors. Furthermore, addressing poor drainage systems, which obscure lane markingsduring heavy rainfall, should be a key focus in improving road safety and sensor accuracy.