This section evaluates the impact of the message generation rules that control the sending of VRU context-awareness messages. First, we obtain the baseline results, where VRU send messages at a fixed rate (i.e., no message generation rules are in place). We then analyze the metrics considering the use of the message generation rules. In both analysis (i.e., with and without generation rules), we consider the same two scenarios and density compositions as shown in
Table 2 and evaluate the impact via the theoretical model and the realistic simulations.
In the following step, we study the variations of the node behavior. Hence, we vary the proportions of moving pedestrians and cycles, as well as the percentage of pedestrians on the street. Such a sensitivity analysis is only performed with the theoretical model, given the limitations of modifying the proportions of nodes at will in the simulation tool because it uses realistic traces [
32].
5.1. Analysis of Baseline VRU Transmissions
First, we analyze the performance of the baseline rule in the two scenarios defined in
Table 2. The baseline rule results in terms of CBR, PDR, and VAP are presented as the blue bars in
Figure 6. In general terms, we observe that all metrics present better performance (i.e., lower CBR, higher PDR, and higher VAP) when evaluated in the low-density scenario (
Figure 6a,c,e) as compared to the high-density scenario (
Figure 6b,d,f). Such behavior is expected, given that the three metrics are dependent on the overall number of exchanged messages.
Our proposed awareness metric (VAP) provides additional, useful information when the frequency is varied. In the low-density scenario (
Figure 6e), we observe that the awareness capabilities offer good performance with VAP values close to 1.0 in both theoretical and simulated results. The difference between beaconing rates is only significant when passing from 1 Hz to 2 Hz, showing an increase of about 1% in the theoretical model and 10% in the simulation results. The high-density scenario (
Figure 6f) gives us more insights into the VAP behavior. Here, the simulation results exhibit a more considerable difference when varying the beaconing frequency of VRU. Following the Baseline-Sim bar, we observe that there is an increase in VAP when the beacon frequency varies from 1 Hz to 5 Hz. When the frequency is increased from 5 Hz to 10 Hz, we observe that the VAP value decreases by nearly 15%.
VAP variations can be explained by observing the formulation in Equation (
9). VAP has a non-linear relation with PDR. In Equation (
9), the exponent stays the same in the 5 Hz and 10 Hz case, while PDR decreases due to the increase in the number of packet collisions (see
Figure 6d). The decrease in PDR is higher in the high-density case; hence, VAP reaches a breaking point where an increase in beaconing rate tends to lower the awareness rather than increase it.
Figure 6f shows that this break occurs at a beaconing frequency of 5 Hz.
5.2. Comparison of Baseline and Message Generation Rules
Here, we present and discuss the results obtained for CBR, PDR, and VAP for the low and high-density scenarios when the message generation rules are in place. The x axis presents the data rates used by VRU to transmit when using each rule. When the MultiTx rule is in place, there is no dependency on the frequency; hence, it is labeled as
—as described in (
1).
In terms of channel load,
Figure 6a,b show the CBR variations as a function of the rules, beaconing frequency, and scenario density. We observe that the most considerable difference is observed between the baseline rule and the PedOnStreet rule. For example, we observe that in the low-density scenario (
Figure 6b), CBR decreases by 20% for the 5 Hz beaconing rate and by 30% for the 10 Hz beaconing rate. In addition, as expected, the higher-density scenario (
Figure 6b) exhibits a higher CBR compared to the low-density scenario (
Figure 6a). The CBR’s decrease when applying the PedOnStret rule is explained by the fact that most pedestrians are on the sidewalk, and hence not transmitting according to the PedOnStreet rule. The MultiTx rule exhibits a difference that changes from nearly 10% when compared to the 5 Hz baseline to 35% when compared to the 10 Hz baseline. The MovVRU rule has a similar behavior as the MultiTx rule since both depend on the VRU movement characteristics. The smaller decrease when using these rules is also explained by the scenario composition, where approximately 90% of VRU are moving.
The other metric that was used to quantify channel load was PDR. We present PDR results in
Figure 6c,d. For both scenarios, this metric exhibits a relatively constant level for the beacon frequencies of 1 Hz, 2 Hz, and 5 Hz. In the 10 Hz frequency, the value of PDR decreases by approximately 15% with respect to its value in the 5 Hz frequency for the high-density scenario (
Figure 6d). In terms of rules, analyzing the 10 Hz beaconing rate in the high-density scenario, we observe an increasing tendency for PDR. When comparing the baseline and the PedOnStreet rule, we observe an increase of about 20% in the high-density scenario. The improvements in these metrics are also observed when using the MovVRU and MultiTx rules with the rest of the tested rates.
Despite the improvements observed in terms of channel load, VRU awareness demonstrates a different behavior. VAP results are presented in
Figure 6e,f for the low-density and high-density scenarios. In the following, we analyze the high-density scenario with a beaconing rate of 10 Hz. The most critical case in terms of VAP corresponds to the one when the PedOnStreet rule is used. When we compare the simulated baseline with the PedOnStreet rule, we observe a reduction of about 50% in VAP; this indicates that an additional 50% of pedestrians is not detected through cooperative communications. When applying the PedOnStreet rule in this scenario, only a few pedestrians, the ones that are actually on the street, are allowed to send messages; all other pedestrians (98%) are not detected.
The VAP decrease is less critical under the MovVRU rule, showing a variation of about 10% when compared to the simulated baseline. A higher number of moving VRU explains this smaller decay. The case of the MultiTx rule is interesting in terms of awareness analysis.
Figure 6f shows that, under the MultiTx rule, VAP increases by about 10% compared to the baseline, reaching a VAP value near a 95%. The behavior can be explained by the formulation of this rule and the improvements exhibited in CBR and PDR. While the MultiTx causes a PDR increase, instead of deferring the VRU transmissions, it changes their transmission rate. The fact that VRU do not stop their communications but change the beaconing rate means that every node has the possibility of being detected, but each node’s message delivery probability is different.
We conclude that message generation rules can have different and opposite effects on channel load and VRU awareness. This is shown most notably with the PedOnStreet rule, where the most significant decrease in the channel load came with the most considerable decrease in VAP compared to the other rules.
5.3. Node Behavior Variation
Using the theoretical model (see
Section 3), we perform a sensibility analysis, varying the percentage of active nodes when applying each rule. The percentage of active VRU is determined by the applied rule and the scenario characterization (proportion of moving VRU and ratio of pedestrians on the street). For this analysis, we consider a transmission rate of 5 Hz for VRU. We study the difference (in percentage) between the values of the selected metrics—CBR, PDR, and VAP—when the different rules are applied and when the baseline case is considered. We study both the case of the low-density and high-density scenarios.
First, we present the results when applying the PedOnStreet rule. In this case, the moving variable is the percentage of pedestrians on the street; cycles are assumed to be always on the street. We vary the percentage of pedestrians on the street between 0% and 100%.
Figure 7 shows the results for the low-density (
Figure 7a) and high-density (
Figure 7b) scenarios. The graphs start at an activation value different from zero because of the presence of bicycles and motorcycles—always active under this rule—as shown in
Table 2.
From
Figure 7, we observe that the pedestrians’ behavior heavily determines the metrics variations. This effect is more evident in the VRU awareness capabilities reflected by VAP. If we focus on the range of active VRU between 40% and 70%, we observe different behaviors for each metric. For the low-density scenario (
Figure 7a), in the case of CBR, we observe a decrease of 10.5% and 5.2% when the percentages of active VRU are 40% and 70%, respectively. Analyzing the same metric in the high-density scenario (
Figure 7b), we observe a decrease when analyzing the same activation percentages.
Figure 7 shows that PDR variations under the same conditions show a similar tendency as CBR, with the most relevant improvements (increases) at lower activation percentages.
Despite the improvements in PDR and CBR, VAP decreases. For example, in the high-density scenario (
Figure 7b), while the CBR values improve, the effect of the rule on VAP is a reduction of 60.6% and 30.3% when the percentages of active VRU are 40% and 70%, respectively. Results for the MovVRU rule (
Figure 8) present the same tendency as that obtained for the PedOnStreet rule, showing an improvement of the channel load metrics while negatively affecting the VRU awareness capabilities. When using the MovVRU rule, the negative effect on VAP could be more severe because even lower percentages of VRU activation may occur. We consider this behavior critical when formulating message generation rules because improvements in terms of channel load—a reduction in CBR and an increase in PDR—could be accompanied by harmful effects on VAP due to the transmission deferral associated with the application of rules.
In contrast with the previous rules, the evaluatio of the MultiTx rule (
Figure 9) differs in its behavior when the node activation percentage varies. From the congested scenario (
Figure 9b), we observe that CBR decreases in the low activation portion while PDR increases. A remarkable difference when using this rule is the behavior of VAP. While in previously tested rules, this metric is severally affected in low-activation scenarios, when using the MultiTx rule, we observe that VAP presents a slight variation compared to the baseline; this means the VAP value is near 1.0, according to
Figure 6f. This can be explained by the fact that even when VRU are stationary, they still send messages but at a lower rate. This means that the set of non-active VRU is not directly discarded from being detected by vehicles.
We chose DSRC as the access technology for the current study mainly because of its extended use and research. Other access technologies of interest, such as C-V2X, may vary on their levels of PDR, CBR, and VAP; however, we believe that our proposed analysis can apply to C-V2X technology with minor modifications to the metrics’ measurements. Despite the difference in the level of congestion and the awareness probability between the two technologies, the results, particularly in PDR and VAP, should follow the same trend [
40,
41]. The authors in [
40] show that PDR, when using C-V2X, exhibits a slighter decrease compared to DSRC when the distance between the transmitter and the receiver varies; however, the decreasing trend is similar for both technologies. The authors in [
41] compare the performance of IEEE 802.11p and LTE-V2X (precursor of C-V2X) using freeway and urban scenarios. The comparison results show that PRR (packet reception ratio), a metric related to PDR, follows a similar trend but with a smaller decrease in the LTE-V2X case for urban scenarios.
5.4. Variation for MultiTx Rule
Among the message generation rules under study, the MultiTx is the rule that shows the best performance in terms of the trade-off between congestion—measured through CBR and PDR—and awareness (VAP). The MultiTx rule varies VRU transmission frequency between 2 Hz and 5 Hz when the VRU are stopped or in movement, respectively (see
Section 3.1). These arbitrary values were defined by the original formulation of the rules [
4]. In this section, we perform a sensibility analysis, modifying the frequency pairs of the MultiTx rule, to investigate the effects of these variations on the performance metrics. We test four new combinations of frequencies (see
Table 5).
Figure 10a–c show the results for CBR, PDR, and VAP for the low-density scenario while
Figure 10d–f show the results for CBR, PDR, and VAP for the high-density scenario. In both low-density and high-density scenarios, the behavior of PDR and CBR worsens when the beaconing frequencies increase. For example, in
Figure 10a,b, CBR increases by 32.3% by using combination
instead of
, whereas PDR decreases by 8.5%. Similarly, in
Figure 10d,e, CBR increases by 39.1% by using combination
instead of
, whereas PDR decreases by 16.8%.
Although the new combination of frequencies have an impact on PDR and CBR, VAP appears almost insensitive to the frequencies variations in the low-density scenario (see
Figure 10c). In the high-density scenario,
Figure 10f shows that the original frequencies (i.e., 5 Hz and 2 Hz) result in a better VAP. In this scenario, we observe a counterintuitive behavior of VAP for combinations
and
: although PDR decreases (see
Figure 10e), VAP shows a decrease for the same combinations, with respect to the original frequencies. Such a behavior can be explained by analyzing Equation (
9).
Figure 10e shows that PDR is lower in
, compared to
, which should negatively impact VAP. However, if there is an increase in the beaconing frequency, the number of opportunities for VRU to be detected increases. These two factors produce the difference in the trend followed by VAP in
Figure 10.
5.5. Summary of Results
This section presented a study of a set of message generation rules based on channel load (CBR and PDR) and VRU awareness capabilities (VAP). As demonstrated in
Section 5.1, the natural behavior of channel load is to grow when the traffic and VRU increase. This section also validated our theoretical adaptation of the model defined in [
30] taking the simulations as reference. In
Section 5.2, we discussed the effects of the various message generation rules on two predefined scenarios. The results showed that the more restricting rules in terms of transmission deferral—PedOnStreet and MovVRU—tend to reduce the channel load more significantly than the MultiTx rule. However, these improvements come with a reduction in VAP due to the more significant percentage of VRU not sending messages to their neighbors. The MultiTx rule proves to be the most suitable in the tested scenarios. Finally, in
Section 5.3, our sensibility analysis showed the benefits and drawbacks that message generation rules could create in different mobility scenarios. The results of this section overall support the notion of defining more dynamic and adaptive message generation rules that successfully function in a more diverse set of conditions.
In the following, we summarize our findings based on the results obtained studying the rules proposed in [
4]. Simple rules can be efficient in computational costs; however, they do not consider several features that can help in the transmission decision making. These rules do not consider the VRU context in their formulation. The context can be used to define the danger levels for different VRU. Moreover, the strict formulation of these rules may be not applicable for dynamic scenarios; indeed, the urban mobility of cars and VRU may lead to an excessive transmission of messages or a reduction in the awareness due to over filtering mechanisms. Several existing contributions emphasize the importance of the VRU intention recognition for the safety of VRU and the prevention of accidents [
42]; VRU intention can be also used to help in the filtering process of messages. In the following, we briefly describe the key existing contributions that could help to develop message generation rules while taking into account the VRU context, intention, and dynamism.
With respect to context, the most common feature is to defer transmission when pedestrians are detected to be indoors; indoor sleeping mode was already considered in [
4] and the WiSafe system [
12]. The authors in [
23] expanded the indoor deferral and included the case when pedestrians are stopped. Using V2C (vehicle-to-cloud) and P2C (pedestrian-to-cloud) communications, the proposed system [
23] also incorporates beaconing management based on the distance between cars and pedestrians. More specifically, vehicles and moving pedestrians send beacons periodically; however, when the system does not detect possible threats to pedestrians, it commands to reduce the beaconing frequency. When the system detects a possible threat, it alerts pedestrians and orders them to change their beaconing frequency to 10 Hz. The authors in [
43] defer the message transmission if any of the following conditions are met: static VRU, VRU indoors, VRU on a vehicle, or VRU in parks or safe regions. There are also solutions that use clusters (e.g., [
22]), where the VRU cluster head exchanges messages with neighboring vehicles and instructs the VRU cluster members to transmit messages only when the speed of vehicles is high.
With respect to the VRU intention, several contributions propose to use visual sensors (mainly cameras), thermal sensors, RADAR (radio detection and ranging) technology, and LiDAR (light detection and ranging) to determine the intention [
42]; the focus was mainly on pedestrians crossing intention and bicycle turning prediction. The authors in [
44] used convolutional neural networks (CNNs) to determine the intention of pedestrians to cross a street. They used the same methodology to determine a cyclist intention to make a turn, assuming that she follows traffic rules to indicate maneuvers (arm signals). The authors in [
45] proposed detecting the intention of vehicles to improve the safety of cyclists against the right turns of vehicles. The authors in [
46] proposed to determine pedestrian intention based on video, using a latent-dynamic conditional random field model, and images including a laser scanner; they use recurrent neural networks, specifically a long short-term memory network with attention mechanism. The authors in [
47] proposed a hidden Markov model, based on the use of 3D positions and displacements of 11 joints located along the pedestrian body, to determine pedestrian intent.
With respect to dynamic rules, we consider that the dynamic nature of the vehicular environment requires context-aware and adaptive rules. This is the subject of our current research; more specifically, we are developing a decision system, based on machine learning techniques, that allows to dynamically select between DSRC and C-V2X (cellular–vehicle-to-everything) in a heterogeneous network.