In this section, we evaluate the performance of the proposed extensions and new protocols, and compare it with the original FRESH protocol. The goal is to demonstrate the feasibility of using AoI for meaningful purposes in DTN and quantify the obtained benefits/costs.
4.1. Evaluation Setup
For the simulation, the Python programming language was used in a Google Colab environment. All experiments were performed on a PC with an Intel(R) Core(TM) i5-7200U
[email protected] GHz, 6 GB available RAM and Windows 10 operating system. In order to make it easier to reproduce and verify the measurements made, the entire simulation code is available online in the Github repository [
36].
In this work, three different versions of the FRESH protocol are simulated and for each of them the original and the modified version of the protocol are compared with each other by incorporating the concept of AoI, as explained in
Section 3.2 The common elements of all six simulations performed are summarized below:
Initially, in each time slot 1–3 new packets are generated with equal probability and with random nodes as their source and destination. Subsequently, all nodes move according to the Random Waypoint Mobility Model [
37] and for each node its current neighbors are found (it is checked if it is more or less than a predefined proximity radius value, and the Euclidean distance is used to measure the distance) and the encounter vectors are updated accordingly. For all simulations, the common assumption made is that in each time slot, each node can send at most one of the packets that it currently has stored in its buffer. More specifically, for the three versions concerning the original FRESH, for each node one of the packets it owns at that moment is randomly selected, while for the three modified versions of FRESH, for each node it is chosen to transfer its packet to the network that has the highest AoI value at this time, in order to make use of the property of Age of Information (the older the age of a packet, the more critical it is to be transported in the network and delivered as soon as possible to the destination). Afterwards, each node in each of the versions of the algorithm chooses according to separate criteria what to do with the packet it chose in the previous step. Thus, it applies a certain reasoning in order to find the next hop of the packet to transmit at this time. At the end, it delivers this packet to the node it found from the above process and after this has been implemented by all nodes in the network, all packets that have not yet been delivered or dropped increase their age by 1. In the three versions of the Modified Protocol, if the age of a packet exceeds the threshold value, the respective packet is dropped from the network.
The three variants of the FRESH protocol, which are simulated both in their original version and in the modified version with the introduction of the AoI variable, are the following:
- (i)
Relay node selection based on neighbors: in this variant, the node selects the relay node of the packet it has selected based on its current neighbors at the link layer. More specifically, if the destination of the packet belongs to the set of its neighbors at this time, then this is the best case, as it delivers the packet directly to its destination. Otherwise, if the destination does not belong to the set of its neighbors at this time, it looks at the encounter vectors of its neighbors and delivers the packet to the neighbor that had the most recent encounter with the destination. In case it has no neighbors at all, it keeps the packet stored in its buffer. This variant is hereafter denoted as FRESH-neigh or Modified-neigh depending on whether the original FRESH protocol is used or the one modified by introducing the AoI metric.
- (ii)
Relay node selection based on “best near anchor” node: in this variant, when each node chooses the packet to forward in the network, it forms a circular ring of a certain radius centered on itself. Then, it searches for the nodes residing inside the ring it has just formed (we call these nodes from now on “near anchor nodes”) and then, it searches if there is a node close to it that has a more recent encounter than its own of the encounter with the destination of the packet. If no near anchor node satisfying this property is found, it forms a new larger ring and repeats the process until at least one near anchor node is found satisfying this property. In the present implementation, an initial ring radius equal to 10 has been chosen and if no node is found that satisfies the aforementioned property, the radius is doubled at each step. This variant is hereafter denoted as FRESH-near-v1 or Modified-near-v1 depending on whether the original FRESH protocol is used or the one modified by introducing the AoI metric.
- (iii)
Relay node selection based on “first near anchor” node: this variant is exactly the same as the above with the difference that here the source node only needs to find one near the anchor node that has an encounter time with the destination more recent than its corresponding time with the destination. On the contrary, in the above variant, when a near anchor node is found that satisfies this property, all the near anchor nodes of the source (with the specific ring radius) are searched and among them the "optimal" one is selected, i.e., the nearest anchor node that has the most recent encounter time with the destination compared to all others near anchors. This variant is hereafter denoted as FRESH-near-v2 or Modified-near-v2, depending on whether the original FRESH protocol is used or the one modified by introducing the AoI metric.
4.4. Cumulative Comparison
In this section, the main results of the above simulations are presented for the three versions of the modified protocol proposed in the work and the three versions of the original FRESH protocol that were studied. The two metrics that are examined are the throughput and the average age of the network packets. The execution time of the simulations is also studied. Initially, in order to analyze the effect of the number of time slots on the simulations,
Figure 5 is constructed, capturing the dependence of the throughput on the change in the number of time slots for the four versions of FRESH (where each node delivers its packet to some “near anchor”).
It is noted that the ratio of the Bound value for the AoI for the two modified versions of the FRESH protocol to the number of time slots selected in each case equals 7% for the first two time slots values, 4% for the price of 5000 time slots, 2% for the price of 10,000 time slots and 1.33% for the price of 15,000 time slots. From the above graph, it is possible to extract the number of time slots above which the so-called “transient effect” has completely disappeared, resulting in the throughput tending asymptotically to a constant value. For all four protocols, it seems that for a number of time slots greater than 5000, the throughput does not change significantly, so it is safe to conclude that the value of 5000 time slots is sufficient to eliminate the transient phenomena. Furthermore, for all time slots values, the Modified-near-v1 and FRESH-near-v1 protocols demonstrate the best performance, since in them, by definition, nodes search for their optimal “nearest anchor node” and not just the first better that they encounter, therefore the chances of successful transmission of the packets are increased in these two protocols. In fact, it is observed that in 4/5 cases, the Modified-near-v1 protocol outperforms the FRESH-near-v1 even for a while, a result which certainly needs justification, as the modified versions of the FRESH protocol were proposed in this paper in order to improve the packet age domain and not (necessarily) the throughput, a result which is confirmed by conducting many of the same simulations and averaging the results. Furthermore, we should not forget the dependence of the throughput on the AoI Bound in the two versions of the modified protocol proposed in the work. A clearer conclusion about this dependence is drawn below by citing a related graph. However, a similar superiority is observed in the Modified-near-v2 protocol over FRESH-near-v2 in the field of performance.
Similarly, in
Figure 6, we demonstrate the dependence of the average age of the packets with the change in the number of time slots for the four protocols.
From this figure, the superiority of the two modified versions of the FRESH protocol over the original versions is evident. More specifically, especially for the large values of the number of time slots, the average age of the packets in the two FRESH-near protocols is almost 30-fold higher than the corresponding age of the packets in the two Modified-near protocols. Therefore, choosing the modified protocol over the original one is an effective option, since the difference in throughput is almost insignificant, while on the other hand, the benefit in packet age is considerable.
Another outcome regards the effect of throughput with respect to how dense the DTN network is, i.e., with respect to the number of its nodes. This effect is depicted in
Figure 7.
It can be observed that for all protocols the throughput decreases almost exponentially with the increase in the network density. This result is expected, as with the increase in the number of nodes, the available “near” nodes also increase, making it more difficult to transmit a packet to its destination. This observation is directly related to a well-known and general result of P. Gupta and P. R. Kumar [
38] that the throughput is
bits per second for each node for a destination that is not too far away, assuming that
n nodes are optimally placed in a disk of unit area, traffic patterns are optimally assigned, the range of each transmission is optimally chosen, and each node is capable of transmitting at
W bits per second. In addition, it is observed that the best performance is again achieved by the two protocols that choose their “nearest anchor node” as a relay node, while again there is a small superiority of the modified protocols over the original ones even in the throughput part.
In
Figure 8, the behavior of the average age of the packets compared to the network density is similarly shown.
Again, the overwhelming superiority of the two modified protocols over the two original ones is evident in the part of the average age of the packets, while overall regarding the part of the network density, it is concluded that the use of the modified protocol FRESH on the one hand does not significantly affect the part of throughput, on the other hand it offers a significant gain in the AoI of the packets, which here are much fresher than the corresponding packets in the original FRESH protocol.
At the same time, an interesting observation to be made is whether the two metrics considered (throughput and average packet age) are affected by the AoI Bound in the modified protocol. Intuitively, if the threshold value for the AoI is reduced enough, on the one hand the packets have a younger age, and on the other hand this leads to more drops, therefore possibly to a reduction in throughput. The figures below (
Figure 9 and
Figure 10) show the effect of AoI Bound on throughput and average packet age for the three modified versions of the FRESH protocol. It is noted that in all cases, the number of time slots is equal to 5000.
As expected, lowering the AoI Bound leads to a decrease in throughput and average packet age in all cases. It is observed that the reduction of the average age is steeper for the Modified-near-v1 and Modified-near-v2 protocols than for the Modified-neigh protocol. At the same time, it seems that in terms of throughput, for all protocols, reducing the AoI Bound up to the value of 100 does not cause serious reductions in throughput, while there is a noticeable reduction when the Bound is set equal to 50. Possibly, the value of 100 is a balance point (for a number of time slots equal to 5000). Therefore, it can be concluded that the reduction of the upper bound of the AoI up to the value of 100 (for the number of time slots equal to 5000) does not significantly change the throughput; therefore, it is considered a profitable option, as it achieves a very sharp reduction in the average age of the packets. The throughput is almost unchanged of the AoI Bound up to the value 100, which is justified by the fact that for large barrier values (800, 600) we have fewer packet drops but much more remaining packets in the system, while for smaller barrier values (200, 100) we have more drops but much less remaining packets, with the result that the final throughput of the system remains almost unchanged, as the number of successfully transmitted packets remains almost constant.
Finally, an undoubtedly important metric in any kind of simulation is the time performance (execution time). More specifically,
Figure 11 shows the time required to complete the simulations for the protocols Modified-near-v1, Modified-near-v2, FRESH-near-v1 and FRESH-near- v2 for various values of the system parameters.
Understandably, in the majority of cases, the near-v2 version requires less execution time compared to the near-v1 version in the respective protocol, since there each node only searches for one best “neighbor” and not the best “neighbor”. In addition, it is observed that the increase in the number of time slots brings about a much smoother increase in the execution time than the corresponding increase brought about by the doubling of the number of nodes in the network, which could be characterized as almost exponential.
Furthermore,
Figure 11 shows that the simulation time in the instance (100, 10,000) is reduced compared to the previous simulation (500, 5000), despite ostensibly being a more complex problem. Regarding this, we should note that we have taken a discrete-event simulation approach for the evaluation of the systems under investigation, which means that in each time slot considered, we account for all events taking place among nodes in the system. The two main parameters of the technique that significantly affect the execution time of the simulation are the size of the DTN network (i.e., practically the number of network nodes) and the number of time slots that will be selected for each simulation. Each simulation produces events between nodes at random, for the duration of the simulation, i.e., the number of slots simulated. Between the above two scenarios, one parameter of the problem increases in value (the number of time slots), while on the contrary the other parameter of the problem decreases in value (the density of the DTN network). Indeed, it is clear that the latter is significantly reduced (sub-fivefold), while the number of time slots is doubled. Furthermore, the number of events taking place in each time slot depends on the number of nodes of the scenario, since the events essentially represent interactions between nodes (and the more the density of the network, the more interactions will emerge on average). Thus, the complexity in the simulation can be essentially attributed more to the developing events in each time slot that require proper handling in each time slot loop, and lesser to the time slot loops. Experimentation confirmed that the increase in the network density leads to a more rapid increase in the execution time than the corresponding increase in the number of time slots (corresponding observations also apply in case of a decrease in these two parameters).