1. Introduction
Recently, most automobile companies have begun the development of intelligent vehicles such as vehicles with Advanced Driver Assistance Systems (ADAS), partially autonomous vehicles, and fully autonomous vehicles. These newly developed intelligent vehicles must satisfy various driving requirements such as safety, comfort, and fuel efficiency.
The foundation of an intelligent vehicle is its ability to understand the environment around it [
1,
2]. For this purpose, a variety of internal sensors such as ultrasonic sensors, RAdio Detection And Ranging (RADAR) sensors, LIght Detection And Ranging (LIDAR) sensors, cameras, and Global Navigation Satellite Systems (GNSS) are installed in intelligent vehicles. However, there are two limitations to understanding the environment which depends only on these internal sensors: the absence of integrity and the perception range. First, the integrity of the information acquired from internal sensors is not guaranteed; the sensors are sometimes vulnerable to environmental noises and are limited by the fact that high-performance computing units cannot be used in vehicles due to their high price and the excessive electric consumption. These problems make it so that the sensors cannot guarantee the complete tracking of the sensing information and real-time performance. In addition, these sensors have limited perception ranges, and the sensors cannot measure target objects located outside of its Field of View (FoV) or target objects occluded by other obstacles.
In order to solve these problems, a High-Definition map (HD map) have been widely researched [
3]. The HD map is a precise and detailed map that stores landmark features which are essential to intelligent driving. Though the composition of the HD map differs according to the map providers (HERE [
4] and TomTom [
5]) or the purpose of the map (localization [
6,
7,
8,
9,
10], perception [
11], and planning [
12]), the HD map generally includes various landmark features (such as road geometry [
13], road maneuver [
14], lane [
15,
16], road surface marking [
17], and traffic control units [
18]).
The use of the HD map improves several functions of the intelligent vehicles: localization, perception, and planning. First, the matching relation between the HD map and the perception measurements improves the accuracy and reliability of vehicle localization without the need for high-cost sensors [
6,
7,
8,
9,
10]. Next, when the position and heading of the vehicle are known precisely by the vehicle localization, the features in the HD map can be used as virtual sensors in the intelligent vehicle. For example, the traffic sign information in the HD map can be “sensed” by the intelligent vehicle without using the perception sensors. In addition, in order to accurately recognize continuously changing information such as the state of a traffic light in real-time, information such as the position and size of the traffic light can be transmitted to the recognition system through the HD map [
11]. Finally, the HD map can be used to generate the local routes and global routes to destinations in the path planning of the intelligent vehicle [
12].
In order to generate the HD map used in intelligent vehicles, professional mapping vehicles equipped with Mobile Mapping System (MMS) are mainly used through three processes [
19]. First, the mapping vehicle equipped with high-performance position and perception sensors travels along target routes in order to acquire the mapping data (data acquisition). Next, the features acquired from the mapping vehicle are accumulated based on the trajectory of the vehicle on the map according to the types of the features (data accumulation). Finally, the features in the map are manually refined and confirmed (data confirmation).
The constructed features in the HD map for the intelligent vehicles are generally managed by the layer-based map management system, which has some advantages in the application for the intelligent vehicle. Editing of the HD map such as addition, deletion, or correction, is efficient because it is easy to visualize and access the map database. In addition, the network bandwidth can be saved by downloading only the required layer, when the map data are downloaded to the intelligent vehicles. Finally, it is easy to support old vehicles, because the formats of the layers required by the vehicles remain despite the addition of the new layers.
Recently, it has come to be required for the HD map to add new feature layers continuously as a solution for more complicated problems in intelligent vehicles, as the functional requirements for intelligent vehicles have increased. Unfortunately, the new feature layer mapping in the conventional method using mapping vehicles has several problems: costs, human resources, and latency. (1) Since mapping vehicles must be re-driven on all roads to acquire new feature data, it requires substantial costs. (2) Many human resources are needed to confirm the newly acquired feature information. (3) Latency necessarily occurs in the addition of the new feature layers, because it takes lots of the times for few mapping vehicles to acquire all of the necessary data.
In order to solve the problems associated with new feature layer mapping based on several mapping vehicles, we propose a crowd-sourced mapping process of the new feature layer based on multiple intelligent vehicles. The main contribution of our paper is the generation of new feature layers in the accuracy level of the HD map using the existing HD map and crowd-sourced information without additional costs, human resources, or latency. Since the layers can be used in intelligent driving, it is essential for the new feature layers to be highly accurate.
For the new feature layers to be highly accurate, the fact that the crowd-sourced information acquired from intelligent vehicles is not as accurate as information acquired from mapping vehicles must be overcome. In order to overcome the inaccuracy of the information, the paper uses two concepts: matching with the HD map and crowd-sourced data.
- • Matching with the HD map:
By considering the matching between the sensor information and the existing features in the HD map, new feature layers are generated with increased precision.
- • Crowd-sourced data:
Combining many new feature layers acquired from many intelligent vehicles solves the accuracy problem that arises when using inaccurate sensors.
The results of the proposed algorithm were evaluated by simulations and confirmed by the experiments in the real driving environments.
In order to explain the mapping of the new feature layer based on two concepts, this paper is organized as follows.
Section 2 presents a system architecture of the proposed system.
Section 3 explains the new feature layer mapping in an individual intelligent vehicle.
Section 4 describes the integration of the new feature layers acquired from multiple intelligent vehicles in a map cloud system, and
Section 5 evaluates the proposed system through simulations.
Section 6 explains the experimental results. Finally, the conclusion is presented in
Section 7.
4. Integration of New Feature Layers in a Map Cloud
The GraphSLAM algorithm generates a new feature layer on the driven roads whenever the vehicle is stopped temporally or at a parking lot. The generated new feature layer is uploaded to the map cloud through the mobile network system. This approach can reduce the network bandwidth because the transmitting data are compressed from the mapping data to the new features in the layer through the GraphSLAM algorithm.
Although the HD map is considered in the estimation of the vehicles, the crowd-sourced new feature layers uploaded from multiple intelligent vehicles may have errors in actual positions of the new features due to the inaccuracy of the low-cost sensors. In order to minimize the errors of new features, the RLS algorithm ([
23]) generates an integrated new feature layer by combining the crowd-sourced new feature layers from multiple intelligent vehicles. Since the RLS algorithm recursively estimates the integrated layer whenever the crowd-sourced layers are entered, the algorithm is suitable for integrating the crowd-sourced layers into the integrated layer.
The values and covariances of integrated new features must be initialized to apply the RLS algorithm.
The
x and the
P are the state and the covariance of the feature in the integrated new feature layer, respectively. The
y refers to the new features estimated in each vehicle. A firstly-uploaded new feature
is used to initialize an integrated new feature. The initial covariance of the integrated feature is determined by the noise covariance
R of the perception sensor equipped in the intelligent vehicle. For the purpose of merging the crowd-sourced layers into the integrated layer, each measured new feature is associated to the integrated feature using the distance-based clustering algorithm. The associated new feature updates the integrated feature using the RLS algorithm.
The Jacobian matrix of the measurement model H is an identity matrix because the measurement y and the estimated states x are the same. Entering futher measurements into the RLS algorithm reduces the covariance and makes the integrated map more accurate.
5. Simulation
A simulation analysis was performed in order to evaluate the crowd-sourced mapping process of the new feature layer with the HD map and to analyze the effects of the noises of commercial sensors on the new feature layer mapping. In the simulation, a beacon layer is generated based on the crowd-sourcing mapping with the HD map which has a lane layer only. The beacon can be modeled by single points which have two-dimensional position information. The reason why the beacon is selected for the new features is that it is easy to intuitively understand the process of the new feature mapping algorithm. Not only the beacon but also other parameterized data can be generated as the new feature layer. On the other hand, the lane in the HD map can be modeled by polylines which have consecutive points to represent lines. Since the lane information exists on most of the actual roads, it is selected as the HD map in the simulation. Various feature types as well as the lane can be used as the HD map.
In order to perform the simulation, a test vehicle is equipped with a high-precision Global Navigation Satellite System/Inertial Measurement Unit (GNSS/IMU) with
m positioning errors, in-vehicle motion sensors, and a low-cost GNSS with
m positioning errors. To estimate new features, the mapping data consists of vehicle motion data (yaw rate and speed), a low-cost GNSS, and virtual noised lanes and beacons. The virtual lanes and beacons replace the actual features for the purpose of analyzing the effects of the sensor noises. The test site is UTAC CREAM in France, as shown in
Figure 4.
5.1. Simulation of Sensors
The GNSS/IMU can measure the precise pose of a vehicle. The precise pose can be used to simulate the information which would otherwise be gathered by low-cost perception sensor information. First, a true beacon layer and a true lane layer need to be pre-defined in the WGS84 coordinates, as shown in
Figure 4. The true beacon layer plays two roles in the simulation: (1) the source information to generate noised beacon measurements, and (2) the ground truth information to evaluate the beacons estimated by the proposed algorithm. On the other hand, the true lane layer also plays two roles in the simulation: (1) the source information to generate noised lane measurements, and (2) the existing feature layer in the HD map for matching with noised lane measurements. Second, when the test car is driven in the test site, the global positions of the beacons and the lanes within the detection range of the car can be inferred from the two true layers based on the precise GNSS information, as represented in
Figure 5. Next, the beacons and the lanes in the WGS84 coordinates are converted in the vehicle coordinate so as to simulate the perception information. Finally, Gaussian white noises are added to the computed precise perception information to simulate the noised beacons and the noised lanes which can be measured by low-cost sensors. The perception sensor noises added to the sensor measurements are designed as functions of the noise level
and
in
Table 1. Basically, the two noise level parameters are initialized as 1.
To match between the simulated lanes (red) and the lanes in the HD map (blue), a generalized iterative closest point (GICP) algorithm is applied [
26]. Since the GICP algorithm matches the source points with the target points, the simulated lanes and the lanes in the HD map are sampled to the points at intervals of
m. The matching relation between two lanes can infer the transformation matrix between the initial pose and the estimated pose.
5.2. Results of Simulation
The results of the new feature mapping algorithm in the simulation are evaluated using the Euclidean distance errors between the estimated beacons and the true beacons. In order to analyze the effects of the HD map in the algorithm, the simulation is split into two cases: new feature mapping without the HD map and new feature mapping with the HD map. Next, the results of map integration are computed so as to analyze the effects of the number of measurements.
The distance errors of the beacons are shown in
Figure 6. The green bars in
Figure 6 are the errors of the beacons generated by the new feature mapping algorithm without the HD map. The mean and the standard deviation of the errors without the HD map are
m and
m, respectively. The blue bars in
Figure 6 represent the errors of the beacons generated by the new feature mapping algorithm with the HD map. The mean and the standard deviation of the errors with the HD map are
m and
m, respectively. The mean and the standard deviation of beacons considering the HD map are smaller than those of the beacons not considering the HD map, because matching information with the HD map is used to estimate the positions of the beacons. In other words, the beacons can be more precisely estimated when considering the HD map.
For simulating the map integration, the crowd-sourced new feature layers update the integrated new feature layer using the RLS algorithm. In order to evaluate the accuracies of the beacons along the number of measurements, the Euclidean distance errors of the integrated beacons are analyzed. The means of the errors of the integrated beacons are represented along the number of the measurements, as shown in
Figure 7. When the number of the measurements is one, the mean of errors of the beacons is
m. On the other hand, when the number of the measurements is 6, the mean of errors is
m. When more measurements are used, the means of the Euclidean distance errors further decrease.
5.3. Analysis of Sensor Noises
To simulate the errors associated with the low-cost perception sensors, the beacon simulator and lane simulator are used in the simulation. To analyze the effects of low-cost perception sensors, the means of the Euclidean distance errors of beacons are analyzed by changing the perception sensor noises.
While the noise level
l of a sensor is changed from
to 2, the noise level
l of the other sensor is fixed to 1 for analyzing the effect of the noise of the target sensor. After the noise level of the target sensor is selected, beacons are estimated using the HD map-based new feature mapping algorithm and the RLS algorithm using crowd-sourced data. The beacons are evaluated by the true beacon map. The means of the errors of all beacons are shown in
Figure 8. The blue line represents the change of the mean of the errors along the change of
at
, while the orange line represents the change of the mean of the errors along the change of
at
. When the noise level of the lane sensor increases, the means of errors of the beacons increase. Since the noised lanes have different shapes with the HD map, the rotation and translation matrices computed by the GICP algorithm are more incorrectly estimated along the more noised lanes. The most significant effect on the result is the noise of the beacon sensor. When the noise of the beacon sensor increases, the estimated beacons become inaccurate, because the noised information is directly combined with the estimated positions of the beacons.
7. Conclusions
This paper proposed a crowd-sourced mapping process of a new feature layer to reduce the costs caused by re-driving mapping vehicles and the delays of the addition of new feature layers. The process is performed in individual intelligent vehicles and the map cloud.
1. The mapping process applies the HD map-based GraphSLAM algorithm. Since the matching information between features acquired from perception sensors and features in the existing HD map is considered, the process increases the reliability and accuracy of the new feature layer. In addition, the new feature layer has high consistency with the existing HD map.
2. In order to compensate for the errors of the low-cost sensors, the map cloud integrates crowd-sourced new feature layers based on the RLS algorithm. Since the algorithm recursively estimates the integrated feature whenever each new feature layer is inputted from each intelligent vehicle, the algorithm is effective in integrating individual new feature layers into the integrated layer.
3. Simulations were performed to analyze the effects of the sensor noises caused by the low-cost sensors. Experiments in real driving environments evaluated the performance of the crowd-sourced mapping for the new feature layer. In addition, the localization performance with the new feature layer was evaluated by comparing it with the localization performance without the new feature layer.
This paper presents the crowd-sourced mapping process of the new feature layer using multiple intelligent vehicles. The approach has limitations in that the performance of the algorithm is largely affected by the performance of the landmark feature detector. In order to overcome the dependency of the detector, the authors plan to research the crowd-sourced mapping process based on the signal level features.