1. Introduction
Pedestrians are a vulnerable group in road traffic accidents. The Annual Statistical Report for Road Traffic Accidents in China showed that pedestrians accounted for 27.11% of the total number of traffic fatalities and 16.72% of the injured in 2017 [
1]. Further, pedestrian accidents have also been recognized as an important challenge for emerging autonomous driving technologies. In March 2018, a pedestrian died in a collision with a self-driving Uber vehicle in Arizona, USA, which was the first pedestrian death in the world caused by an unmanned vehicle. Some researchers have found that the majority of traffic accidents could be prevented if a vehicle was able to recognize risk one second in advance. Therefore, risk assessment is of great significance for pedestrian collision avoidance in intelligent vehicles.
Many researchers have attempted to address this issue. Generally, risk assessment includes two types of typical methods in the post-crash stage and pre-crash stage. In the post-crash stage, road traffic risk assessments are generally measured on the basis of historical accident data [
2,
3] and represented as the indexes of road safety or relative accident rates obtained using statistical models such as accident regression analyses [
4,
5]. For example, Hojjati-Emami et al. [
6] determined the causes of pedestrian–vehicle accidents on roads using a fault tree model. However, slight collisions are generally not included in traffic accident records, and traffic risk analysis methods that are based on previous accident data have low adaptability and accuracy limitations. In the pre-crash stage, classification is proposed on two levels—physics-based models and potential field-based theories—which will be introduced as following. Physics-based models are risk assessment methods which focus on different longitudinal lateral vehicle motion scenarios and directions. They are most commonly used in longitudinal analyses, such as the time-to-collision (TTC), inverse of time-to-collision (TTCi), deceleration rate to avoid a crash (DRAC) [
7,
8,
9], and lateral analyses, such as the car’s current position (CCP), time-to-lane-cross (TLC), and the variable rumble strip (VRBS) [
10,
11,
12]. However, in real traffic environments, traffic risks are continuous variables that cannot be artificially classified into longitudinal or lateral directions.
Potential field-based methods for road traffic risk assessments, which have been widely used in obstacle avoidance and path planning for intelligent vehicles in recent years, are able to overcome the aforementioned problems because of their advantages in terms of environmental description. In 1986, Krogh et al. [
13] firstly applied the potential field theory to the perception of the surrounding environment of a robot. Gravity was used to indicate the walkable area, and repulsive force indicated the non-walking area. This was mainly used for the path planning decisions of a robot. In 2012, Hsu et al. [
14] introduced the concept of the gravitational field to establish a car-following model based on Krogh’s research. Gravity and repulsive forces were applied to model the lateral and longitudinal behavior of the vehicle, and data calibration was performed from simple experiments. In 1994, Reichardt et al. [
15] compared the vehicles in a traffic environment to virtual electronic vehicles, and built a vehicle-centered power field combined with vehicle information, obstacle information, and traffic rules. The system was used to realize the horizontal and vertical control in intelligent vehicles on expressways. Simulation experiments were conducted to verify the validity and practicability of the method. However, although the theory was in line with the actual situation to a certain extent, there were still many realistic scenarios that could not be matched. To solve the problem, Tsourveloudis et al. [
16] combined the electrostatic potential field theory with two-layer fuzzy logic reasoning. The environmental road network was analogized to a resistance network, in which each vehicle generated an electrostatic potential field. The optimal path of the vehicle was planned by the current in the network, thereby realizing the automatic path planning of an intelligent vehicle in a dynamic environment. Sattel et al. [
17] combined potential field with elastic order from the field of robotics. The elastic order was used to calculate the minimum path in the potential field, in order to plan a vehicle path and avoid falling into the local optimum. The potential field was used to model the lane centerline and lane boundary for collision avoidance. The theory could effectively distinguish between static objects and dynamic objects, and also adapted to the operating characteristics of different drivers through parameter control.
In 2013, Matsumi et al. [
18] examined an autonomous collision avoidance system based on the electric braking torque in electric vehicles. In the system, the braking maneuver intensity was determined using potential field theory, which considered the potential hazards because of occlusions in the intersection. Raksincharoensak et al. [
19] proposed a virtual spring model that connected a vehicle and pedestrian, to determine the potential field in unsigned intersections with poor visibility. In the same year, Ni Daiheng et al. [
20] proposed the field theory of a microscopic traffic model, which was mainly applied to the car-following model. The difference between these theories was that Professor Ni forwarded the vehicle along the lane as a free fall of the object. The vehicle was subjected to the gravitational force brought about by the advancement, the repulsive force brought by the surrounding vehicles, and the repulsive force caused by the difference between the current speed of the vehicle and the driver’s desired speed. Hence, a potential field was formed. The vehicle moved along the lowest point in the potential field map. The theory effectively conforms to the behavioral habits of real drivers; however, it is limited to scene restrictions and does not apply to scenes in which pedestrians interact with cars. Based on Ni Daiheng’s theory, Wang et al. [
21,
22] constructed a unified Driving Safety Field (DSF) model that included the following: (1) a potential field, which was determined by static objects on the roads, such as a roadblock; (2) a kinetic field, which was determined by the moving objects on the road, such as vehicles and pedestrians; and (3) a behavior field, which was determined by the individual driver characteristics. Rasekhipour et al. [
23] applied the artificial potential field method to path planning, obstacle avoidance, and road traffic risk assessment in intelligent vehicles. Different potential functions were assigned to different types of obstacles and road structures. Path was planned based on these potential functions, and vehicle collision avoidance was achieved combined with the planning path and vehicle dynamics constraints. However, the calibration of the parameters is very complicated. To simplify parameter calibration, Zheng et al. [
24] developed a model based on equivalent force, which calculated the road traffic risk field in different directions. In all these researches, the road traffic risk was quantified and presented as a distribution in a potential risk map, in which the value of potential energy denoted the road traffic risk level. In this study, DSF is chosen for risk assessment because of its ability to consider two-dimensional risk.
However, the potential field-based method for vehicle risk assessment relies on sensor detection information and causes unnecessary or lacking braking occasionally. It seriously hinders traffic safety. For example, a pedestrian beyond warning range may come into the range and have a crash soon after. A pedestrian who makes the safety assistance system emit an alarm may take no risk for a vehicle because of the change of relative motion. Furthermore, vehicles and two-wheelers are limited by kinematics and dynamics and could not change direction and velocity immediately. Pedestrians are different from them in this term. A pedestrian could do a series of actions consisting of starting, changing his/her direction and stopping in a very short time step. Moreover, frequent brakings relying on pedestrian information from sensor detection would reduce passengers’ driving comfort and increase energy consumption. Therefore, it is meaningful to take pedestrians’ motion intentions into consideration and predict their trajectories for active risk assessment.
Contributions of this paper could be as follows:
- (1)
A system of risk assessment for pedestrian–vehicle collisions is proposed.
- (2)
The intention of pedestrian crossing is considered in trajectory prediction model.
- (3)
The predicted information of pedestrian is applied to DSF for risk assessment.
The remainder of this paper is organized as follows. In
Section 2, intention inference process based on Dynamic Bayesian Network (DBN), and the motion prediction with particle filtering are employed for pedestrian trajectory prediction. In
Section 3, a modified DSF which combines with prediction is developed for risk assessment. In
Section 4, a data collection and parameters calibrations are introduced, after which the effectiveness of the proposed model is verified by conducting Monte Carlo simulation experiments 1,000 times in pedestrian–vehicle collision scenarios.
Section 5 provides the discussion and concludes the paper.
2. Overview of Proposed System
In this paper, the system is divided into five parts. A block diagram of overview system is shown in
Figure 1.
The first block is an input module. Information on pedestrians, vehicles, and road environments is collected from sensors including LiDAR (Light Detection and Ranging), monocular cameras, IMU (Inertial Measurement Unit) and CAN (Controller Area Network). A data-collecting scenario is set at unsignalized roads section in campus, where the pedestrians may walk/cross frequently and pedestrian–vehicle accidents are more likely to occur.
The second block is the pedestrian trajectory prediction module. A pedestrian’s motion features and environmental features are incorporated in a graphical model of DBN, and the crossing-intention of a pedestrian can be estimated through the probabilistic reasoning. A particle filtering is used to forecast a pedestrian’s trajectories, combining with the predicted motion intention.
The third block is vehicle trajectory prediction. Since the research of this paper serves for intelligent vehicle decisions, vehicle trajectories are predicted under the assumption that the vehicle is driven at a constant velocity and invariable yaw angle. Speed changing from decision changes under future pedestrian–vehicle interactions is not considered.
The fourth piece is risk assessment. A modified DSF model was proposed. Predicted trajectories of pedestrians and vehicles are set as input. Behrman equation is defined considering the uncertainties of predicted positions.
Finally, the pedestrian risk value is output to serve the intelligent vehicle decisions.
3. Pedestrian Trajectory Prediction Model
Many novel methods have been proposed for pedestrian behavior modeling and prediction. Mamun et al. [
25] studied the effects of pedestrian crossing motion using pedestrian safety interventions and demographic data. Zhou et al. [
26,
27] used social force model to describe pedestrian collision avoidance patterns, and decision process with considering the impact of PFG (pedestrian flash green) signal on pedestrian crossing motion. In other methods, pedestrian prediction is based on many predecessor features: relative position of pedestrians, speed, pedestrian posture, and depth parallax map and road structure [
28,
29,
30]. However, in most studies, neither pedestrian nor environmental information have been considered. Therefore, this paper considers pedestrian condition, ego vehicle condition, and road environment, with pedestrian trajectory prediction being divided into an intention inference process and a motion prediction process.
3.1. Intention Prediction
A DBN is employed to combine qualitative intentions with additional context information. The intent-related position, motion, and feature details are regarded as hidden variables related to observables, which are variables collected from vehicle equipment.
As shown in
Figure 2, hidden variables are selected as pedestrian intent node
, which has two possible intentions: crossing the road and not crossing the road. Node
is set as an aggregation of the observables from vehicle sensors, and node
is set as an aggregation of the hidden variables in a pedestrian’s mind [
31].
where
is the pedestrian velocity orientation,
is the longitudinal distance from pedestrian to ego vehicle, and
is the lateral distance from pedestrian to the vehicular road curb. In the hidden variable aggregation
,
denotes whether a pedestrian needs to cross, which is related to velocity orientation;
denotes whether a pedestrian feels any danger when crossing;
denotes whether a pedestrian is located on a vehicle road region at which it is possible to cross;
, and
are discrete variables; and
,
are continuous variables.
, and
P are independent of each other, have temporal transition, and accord with conditional probability formula. Therefore, the temporal transition for hidden variables
is as follows:
where
,
, and
are the temporal transitions for goal
, risk
, and position
, respectively, from time step t − 1 to t.
The probability relationships
are independent of each other and also accord with the conditional probability formula. Therefore, the relationship between the observed variables
and the hidden variables
could be written as follows:
where
,
, and
indicate the relationships between
, respectively, in the same time step.
accords with a Gaussian distribution with a mean value and a standard deviation ; conforms to a Gamma distribution with a parameter λr; and accords with a Polynomial distribution. The conditional distributions for these observables are chosen, and their empirical distributions and parameters are estimated using maximum likelihood estimation on the training data, which are manually annotated with the context states.
The inference DBN process employs Assumed Density Filtering as the approximate inference method [
31] and is divided into the “predict” and “update” steps. Prior probability
and posterior probability
are then introduced in the inference process, which uses the posterior probability of the previous step
as the basis for the “predict” step, for which the prior probabilities
and
are calculated, after which the prior probability is updated to
combined with the observables.
At every time step, the network is updated with all available observables, from which the posterior probability of crossing function is inferred.
3.2. Motion Prediction
A DBN-based model is used for pedestrian decision recognition, in which it is necessary to know where the pedestrian is going to reach in the following time steps. Therefore, a pedestrian motion model is set up by employing a sample-based particle filtering method, with the approximate Bayesian filtering algorithm based on a Monte Carlo simulation. Different from the Kalman filter method, which describes the system using explicit probabilistic model parameters, the particle filtering uses the sample mean of the particle set to estimate the parameters. That is to say, the particle filtering sets discrete particles to simulate the probability density of the variables. The weighting and recursive propagation of its particle set are also based on the Bayesian criterion.
The priori predicted value of the particle state at time t is calculated on the basis of the pedestrian motion equation.
where (9) to (11), respectively, are the equations for the change in pedestrian position, direction, and velocity;
is the position of the pedestrian’s absolute coordinates at time step
t;
is the direction of motion; and
is a speed variable without direction at time step
t. Therefore, we have
,
, and
.
indicates the direction noise for pedestrians who are maintaining their current direction, so as to obtain
.
Traditionally, Gaussian functions are usually used for importance sampling in the particle filtering, with the particle importance weight being generally updated by comparing the particle states with the observed true values. However, in this research, as there are no observed prediction values,
is set for the weight calculations in consideration of the particle states and crossing probability from the DBN-based intention model.
where
and
denote whether the particle is in the sidewalk region or the vehicle road region;
is the posterior probability for pedestrian crossing-intention;
indicates a road crossing; and
indicates that there is no crossing decision.
After the weight calculation, some particles are discarded and the remaining particles are normalized using the following formula:
Then, depending on weight size, random re-resampling is employed to obtain a posterior update value for the particle states. The updated particle distribution and the expectation of the next time step will be output.
4. Risk Assessment Model
The DSF is a physical field that denotes the influence of objects on driving safety [
22]. In this paper, the objects refer to elements such as vehicles, pedestrians, and cyclists that could collide with vehicles and result in significant losses.
where,
is the field strength vector at
formed by ego vehicle at
that denotes the potential risk to the surroundings of ego vehicle under the prevailing road conditions;
is the virtual mass of ego vehicle;
is the distance vector between the pedestrian and ego vehicle center of mass; and
is the velocity vector of ego vehicle.
is directed along the gradient descent direction of the field strength when away from ego vehicle; that is to say, the field strength increases as
decreases.
where
,
, and
are undetermined constants greater than 0;
is the road width parameter that aims to limit the risk in one lane;
is the angle between the direction of
and the
x-axis; and
is the angle between the directions of
and
. When
denotes the same distance, the field strength increases as
decreases; that is to say, the potential risk increases; and
denotes the gradient descent direction of the field strength
. The center of the field is the center mass of ego vehicle, with the field strength being more densely distributed in the forward direction of ego vehicle.
where
is the virtual mass of ego vehicle;
and
are the weight and velocity;
is the type such as vehicles, pedestrians, and cyclists; and
, and
are constants greater than 0, the values of which are determined by the maximum likelihood estimations of the relationships between velocity and accident damage.
where
is the field force vector at
formed by ego vehicle at
that denotes the potential acting force on the surroundings because of ego vehicle under the prevailing road conditions and
is the virtual mass of the pedestrian at
.
where
has the same gradient descent direction as the field strength and
is the unit field strength descent.
Because of errors in the pedestrian trajectory prediction with time,
is employed as an attenuation coefficient to solve the problem based on the Bellman equation; that is to say, the field force in rear steps is attenuated.
where
indicates the new field force vector based on prediction at
that is formed by ego vehicle at
on the basis of the trajectories in the following predicted steps.