Fog Effects on Time-of-Flight Imaging Investigated by Ray-Tracing Simulations ^†

Abstract

Time-of-Flight (ToF) sensors are a key technology for autonomous vehicles and autonomous mobile robotics. Quantifying the extent of perturbation induced by atmospheric phenomena on ToF imaging is critical to identify effective correction strategies. Here we present an approach that uses optical ray-tracing to simulate the ToF image, while the distance information is recovered by analyzing the optical path of each ray. Such an approach allows, for example, understanding the effects of different ray paths on the ToF image, or testing various retrieval/correction algorithms upon running a single ray-tracing simulation. By modelling several scattering scenarios, we show that ranging errors arise mostly from light backscattered to the sensor prior reaching the scene. Scattering events close to the sensors (<1 m) have the largest influence, therefore strategies capable of filtering out signals from distances shorter than the range of interest can significantly improve the accuracy of ToF sensors.

1. Introduction

In Time-of-Flight (ToF) imaging, global illumination refers to light emitted from the light source of the sensor and returning to the detector after multiple reflections or scattering events within the observed scene. Global illumination is known to induce ranging errors in ToF sensors, due to light travelling through different optical paths and reaching the same sensor pixel. For outdoor applications, fog and other scattering particles located between the sensor and the scene are expected to affect the ranging accuracy of the sensor via (i) scattering of emitted light back to the detector without interaction with the scene (backscattering), (ii) single and (iii) multiple scattering events including interaction with the scene. Backscattering causes an underestimation of the scene distance, while both single and multiple scattering events contribute with an over-estimation of it. The net effect depends on several parameters, including the angle of view of the sensor, the object distance and the parameters of the scattering particles (size, density, absorbance).

Several approaches have been investigated to minimize the effects of global illumination on ToF imaging, including reconstruction algorithms exploiting the sparsity of the light transport, frequency analysis approaches, frequency coding, multiple frequency imaging and phasor imaging [1,2,3]. However, in order to understand the extent of perturbation induced by scattering particles on ToF imaging, and thus to identify the best correction approaches, a systematic investigation of scattering scenarios is useful. Here we apply a recently developed approach, based on ray-tracing simulations, to investigate the extent of perturbation induced by fog on ToF imaging, as well as to identify the ray-paths that contribute mostly to the deterioration of the ranging accuracy.

2. Materials and Methods

Our simulation approach consists of two steps. First, a ray-tracing simulation is performed for a ToF sensor in front of a model scene. In our implementation, the ray-tracing is performed using the Zemax OpticStudio^© software, which already contains a built-in Mie-scattering routine. Second, the full ray list is analyzed by a self-developed Python^TM routine to extract, store and classify the relevant information for each ray (intensity, position at the detector, travelled distance, scattering events). The distance information is reconstructed on a per-pixel basis either according to a four-step phase detection (correlation-based ToF) or directly from the average time-delay of the rays hitting the pixel (direct ToF). The difference between the two evaluation procedures was described previously [4], and the latter is used for this study.

The ray-tracing model is shown in Figure 1. It focuses on a ToF sensor designed for short-distance operation and covering a field of view of 13.2° × 9.9°. The sensor is composed of a source (VCSEL array emitting at 850 nm), and a receiving module, formed by a paraxial lens, a round aperture and a detector. The scene is composed of four cubes with identical surface scattering properties. The surfaces are assumed to scatter all the incident light with a Gaussian scattering profile. The scattering medium is described by a Mie-model, with scattering centers of homogeneous size.

3. Results and Discussion

Figure 2 presents the simulated point cloud (intensity and distance information) without scattering (Figure 2a) and in presence of scattering (Figure 2b–d) for different particle distributions. The average particle sizes and densities for the different distributions are listed in the figure caption and were chosen according to reported fog conditions [5]. The presence of the scattering medium induces a general underestimation of the object distance, together with the appearance of a diffused background noise. The background signal is not spatially homogeneous. More background intensity is observed on the sensor side closer to the source, which is due to the combination of the angular distribution of the Mie scattering and the acceptance angle of the detection unit. For each particle distribution, the relative contribution of different optical paths to the total signal is plotted in Figure 3, and shows that the most significant source of error arises from backscattering of the rays prior reaching the scene.

The effect of scattering on the ToF image is better quantified by the comparison of the reconstructed distance images, calculated from the average optical pathlength at each sensor pixel, in absence (Figure 4a) and in presence (Figure 4b) of scattering centers between sensor and scene. The scattering distribution (r = 10 µm, ρ = 40 particles/cm³) is the same as shown in Figure 2c. The largest errors in the distance measurement are observed on the cube surfaces which form the largest angles with the source and detector system, since these surfaces contribute with less intensity to the object signal. Figure 4c shows that eliminating all rays sampling a sensor-scene distance shorter than 61 cm – corresponding to about one third of the fog layer – allows to significantly reduce the distance error. We note here that these results are obtained assuming that all the light incident on an object is scattered. Any absorption at the objects would result in an intensity reduction of the “direct path” source-scene-detector, while the backscattering intensity remains unaffected. Therefore, the results presented here represent the best possible scenario for ToF sensing in the described scattering environments.

4. Conclusions

In conclusion, we use a recently developed ray-tracing-based approach to simulate the perturbation induced by fog on ToF imaging. Our results show that the presence of scattering centers between the sensor and the scene affects significantly the distance reconstruction already for short distances and moderate particle densities. Backscattering, whose intensity increases with increasing scattering cross-section of the medium, represents the largest source of error. Accordingly, the object distance is mostly underestimated. As most backscattered rays come from the first tens of centimeters in front of the sensor, filtering out signals from such short distances can help improving the ranging accuracy of ToF sensors in scattering environments.