Dynamic Catadioptric Sensory Data Fusion for Visual Localization in Mobile Robotics ^†

Abstract

This approach presents a localization technique within mobile robotics sustained by visual sensory data fusion. A regression inference framework is designed with the aid of informative data models of the system, together with support of probabilistic techniques such as Gaussian Processes. As a result, the visual data acquired with a catadioptric sensor is fused between poses of the robot in order to produce a probability distribution of visual information in the 3D global reference of the robot. In addition, a prediction technique based on filter gain is defined to improve the matching of visual information extracted from the probability distribution. This work reveals an enhanced matching technique for visual information in both, the image reference frame, and the 3D global reference. Real data results are presented to confirm the validity of the approach when working in a mobile robotic application for visual localization. Besides, a comparison against standard visual matching techniques is also presented. The suitability and robustness of the contributions are tested in the presented experiments.

1. Introduction

Standard visual localization methods in mobile robotics have been widely acknowledged thanks to the unequivocal matching of categorical and physical information extracted from visual sensors [1], transferred into images. One well accepted method is feature point matching [2]. However, dealing with realistic applications in mobile robotics implies that certain issues may arise and jeopardize the final localization estimate [3]. In this sense, external noise sources are very likely to affect the visual sensor input, such as non-systematic and non-linear effects. This work relies on a catadioptric sensor represented by an omnidirectional camera. Its main strength falls on the capability to encode wide scenes over 360 degrees around the camera axis. Nonetheless, due to the nonlinearities associated to the geometry of its hyperbolic mirror, it turns to be a visual system liable to suffer from the such type of harmful noise effects.

Under this context, we present a dynamic approach for visual localization sustained by the sensory data provided by the catadioptric sensor, as an omnidirectional camera. The core of the system considers the data extracted from the visual sensor, in order to be fused at each motion step of the robot. Information metrics [4] are computed over the images, so as to feed a regression module, which is also supported by Gaussian processes (GP) [5]. The output data are accumulated and fused all along the trajectory of the robot. This procedure permits inferring a probability distribution for the visual feature matching. Moreover, exploiting the relevance of the Kalman Filter (KL) gain, we are allowed to define a set of predicted feature matching candidates over the next pose of the robot at t+1.

Therefore, the main contribution of this work is a visual localization technique which dynamically adapts to the non-linear noise effects generated by the environment. Such adaptive dynamic is achieved through the proposals for data fusion and inference, at each time step. Furthermore, this approach emerges as a promising alternative to general outlier techniques, which tend to compromise the computational load of the system to work in real time. The validity and suitability of the approach are evaluated in an experimental setup.

2. Catadioptric Sensor

The real system used in this work is represented in Figure 1. Figure 1a shows the Pioneer P3-AT robot, and the details of the catadioptric sensor, consisting of a CCD (Charge Couple Device) camera jointly coupled with a hyperbolic mirror. Figure 1b details the projection model, by which a 3D point, Q, is projected towards the mirror surface on P, and finally addressed towards the focal of the hyperboloid (F). Along its path, it intersects on the pixel frame of the camera, represented by the 2D point p(u,v).

3. Dynamic Matching

The essential basis for the localization estimation is established by the epipolarity constraint, as described in previous works [6]. The image data fusion between poses of the robot is defined as:

X_G = T + RX_t

(1)

being T and R the predicted translation and rotation that relate the current image data, X_t, processed from the pose of the robot at time t, which is finally fused into the 3D global reference system, denoted as X_G. Additionally, the informative inference is stated by the use of Kullback-Leibler (KL) divergence and input into a Gaussian process, whose nomenclature responds to a function f(x) with mean m(x) and covariance k(x,x’). This function is applied to the 3D global data accumulated in X_G.

f(x) = GP[m(x), k(x,x’)]

(2)

Thus f(X_G) returns a probability distribution of inferred feature point matching, namely p(x,y,z). That is to say, the 3D areas from where feature matching is more probable according to the history in the system. It is worth noting that dynamic updates of the current uncertainty of the system are also considered and accordingly propagated through f(x). The last step takes into account the KF gain to predict the modulation of such probability distribution when the robot moves to its next pose in t+1. Figure 2 presents an example of such probability distribution, p(x,y,z), for probable feature matching by means of this dynamic data fusion approach. Please note that the probability data have been projected onto the 2D image frame of the camera, p(u,v), so as to proceed to the final matching. Ultimately, Figure 3 presents a performance example of the dynamic matching proposed in this work. A standard matching [2] (blue) is compared with the proposal (green). Red points indicate the probability distribution for feature point existence, p(x,y,z), projected onto the image pixels, p(u,v).

4. Experiments

An experimental setup has been conducted over a publicly dataset [7], to confirm the validity, suitability and robustness of the presented approach. Figure 4 presents comparison results for the accuracy of the dynamic matching. Besides this, Figure 5 provides localization results generated in a large indoor scenario.