*6.1. Simulation Process: Algorithm Behavior*

After the simulation has been started, at 200 sims the H11 host starts to generate packets of protected flow with destination address of H42. This flow, as we have already mentioned, is called a protected flow because the routers in M-REP are configured to encapsulate and flood its packets around the point of failure on the way to the destination. Therefore, at the time of 210 sims, we simulate the first failure inside area 1, where we turn off the router R14. As the R14 router is on the best path to the destination, the OSPF will begin to flood its OSPF Link-State Advertisement (LSA) updates and will start to converge. However, until the convergence ends, the R12 router quickly detects that its neighbor has failed (they have either direct connection or use the BFD mechanism), and R12 becomes the source router (S router). That is, R12 begins encapsulating the unicast packets of the protected flow into new multicast packets of the (S, G1) pair, since it is configured to protect the flow from 192.168.11.2 to 192.168.66.2. When creating a multicast packet header, the router will use the original IP address as the source address S, i.e., the IP address of sender (H11). As the multicast destination address G1, the router will use the predefined M-REP multicast address (unique and configured for each protected flow); here it is 226.1.1.1. This behavior is shown in Figure 14. The original unicast destination address is stored for future decapsulation in a variable named MREPdestAddress. The multicast packet is then immediately flooded out using the PIM-DM mechanism. As a result, due to the M-REP algorithm modification (the rule of first arrival), the PIM-DM will construct an alternative path around the failure. The new path lead through R11 → R12 → R13 → R16 in this area.

**Figure 14.** Encapsulated EM-REP packet.

At 212 sims, we have scheduled the second failure, in which the connection between R13/R16 will be interrupted. The link is already a part of the alternative route (distribution tree) constructed before. Here we have both, a second error and a protected flow, which is once encapsulated. The moment this failure occurs, R13 becomes the new source router S2. R13 as S2 detects that it is already on the previously constructed repair path of the first M-REP run (according to its record that includes interfaces list for (S, G1) tree). Therefore, for each multicast packets of (S, G1) pair, the router R13 replaces the previous M-REP G1 multicast address (226.1.1.1) by the new one, G2 (226.1.1.2, Figure 15). This event triggers a new flooding process, creating a new alternative path around the second failure. In this case, the path leads through the routers R11 → R13 → R15 of the area 1.

**Figure 15.** Second EM-REP run.

Our enhancement of the M-REP algorithm assumes, that the decapsulation of original unicast packets of a protected flow from the carrier multicast packets is performed on an ABR router. The ABR selection is performed according to the conditions specified in Section 5.2, which in our case is the R16 router. The modification proposed in Section 5.2 stops the flooding of M-REP packets from one area to other areas, except area 1. Router R16 replaces the destination address of the G2 M-REP packets back to the original unicast one (i.e., 192.168.66.2 taken from MREPdestAddress header, Figure 16). This converts the multicast communication back, and the flow from H11 to H42 is routed as unicast again.

**Figure 16.** Restoration of EM-REP packet to original destination address.

The third and final simulated failure is scheduled at the time of 215 sims. The failure represents an error of the R05 router, that is located inside of the backbone area 0. Here, the process of encapsulating and PIM flooding is repeated. When this failure occurs, R01 becomes the next source router S and begins encapsulating packets of the protected flow using the predefined M-REP multicast address. The multicast address, as we mentioned, must be configured for the protected flow. The failure

occurred in a different OSPF area than in the previous two cases. Our proposed solution reduces the flooding from one OSPF area to another, so that M-REP can use the same multicast destination address as in area 1, i.e., the multicast address 226.1.1.1.

Table 5 displays the output from OMNeT++ simulation, which shows how packets are handled in the moment of the R05 router failure, and which destination addresses are used to deliver packets of the protected flow. The M-REP constructs an alternative route that leads through routers R01 → R03 → R04.


**Table 5.** The result of the third failure simulation.

#### *6.2. Evaluation of the EM-REP Mechanism*

The main advantage of the M-REP IPFRR mechanism is that the algorithm does not depend on precomputations and even on the unicast routing protocol used. Respecting these properties, we may argue that the M-REP IPFRR mechanism is unique compared to the analyzed ones, as well as other existing IPFRR solutions. The extensions of the M-REP proposed in this paper solve several limitations that have not been resolved in legacy M-REP proposal.

The enhancement described in Section 5.1 has introduced recovery mechanisms that support fast reroute in the event of multiple and persistent connection and node failures thorough the whole network. In situations when subsequent and recurring errors occur concurrently over time, the EM-REP mechanism encapsulates protected flow into several specific multicast distribution trees, or (S, G) traffic groups. For each multicast distribution group, its router S as the root of the distribution tree encapsulates and floods packets through any of its functional links to all PIM-DM neighbors. Therefore, even if a link or node failure occurs in several places, if there is still at least one possible path from the source S to the destination D, the EM-REP mechanism can find it and use it as an alternative path. This is a unique behavior of EM-REP that provides the 100% repair coverage. This behavior was among other ones simulated by the complex scenario just described. Here we have simulated three consecutive network failures. The simulation results confirmed expected core EM-REP behavior (detection, encapsulation, and flooding), as well as the extended protection against multiple failures, as we proposed in Section 5.1. The EM-REP mechanism has constructed several distribution trees and protects the data of the specified protected flow from multiple failures. The mechanism ensures that all packets of the protected flow were delivered to its destination D.

The second enhancement, which was described in Section 5.2, addresses the issue that in the original M- REP mechanism, the destination host has to be directly connected to a network with a router that performs decapsulation (router D). The extension also reduces a flooding process of M-REP packets in a network with multiple routing areas.

Compared to other existing IPFRR mechanisms, the concept of the M-REP mechanism (as well as its enhanced version EM-REP) brings several advantages in addition to the mentioned ones. Some of its drawbacks are also known. An overview of advantages and disadvantages of the EM-REP mechanism is given in the Table 6.


**Table 6.** Features of M-REP mechanism.

A more accurate comparison of the selected features with other existing IPFRR mechanisms is provided in Table 7 below. As we can see, the EM-REP mechanism is unique in several specific areas.


**Table 7.** Comparison of the innovative M-REP mechanism with existing solutions.

#### *6.3. Time of Repair: Algorithm Speed*

The network recovery time usually consists of two parts. The first part consists of the time in which a router is able to detect the failure of its link, or the unavailability of its connected neighbors. In practice, a specialized protocol is usually used for this purpose. The most used protocol is the Bidirectional Forwarding Detection (BFD), a protocol standardized by Internet Engineering Task Force (IETF) in Request for Comments (RFC) 5880. Using BFD, the router can detect a connection failure with a neighboring node in less than 30 ms, depending on the timer settings. Specifically, the mentioned time of 30 ms can be achieved by setting the hello interval to 10 ms. If no hello message is received from a neighboring node within three hello intervals, the BFD session with that neighbor is declared invalid, i.e., the neighbor is considered unavailable. It is the state of unavailability that subsequently triggers an IPFRR mechanism.

The second part, which defines the recovery time, is the amount of time required to create an alternative path and resume an interrupted communication. This time depends mainly on the speed of the specific IPFRR mechanism (or its algorithm). Current FRR mechanisms operate using a proactive approach. This means that all alternative paths for all possible destination are calculated in advance before the outage itself occurs. These preliminary calculations differ in their computational complexity and time depending on the IPFRR mechanism used. At the same time, different IPFRR mechanisms have different requirements regarding the required space needed to store their results. However, once a failure is detected, the installation and use of a pre-calculated alternative path is immediate. Compared to link failure detection, this time is minimal and negligible.

However, the EM-REP mechanism operates in principle in a reactive manner. An alternative path is created randomly as the result of flooding and pruning mechanisms used by our modified PIM-DM (i.e., EM-REP). The distribution path (single-branched tree), as has been already mentioned, is constructed using the principle of the first arrival of packets, i.e., packets that arrive first on individual router interfaces after the flooding process. This subsequently creates interfaces of the first arrival (PIM-DM RPF ports) and a chain of routers of an alternative path. Simulations show the speed of network recovery achieved by the EM-REP mechanism is comparable, respectively the same, as the network recovery speed achieved through the proactive FRR mechanisms. The main difference is that the EM-REP mechanism does not require preparatory calculations or additional router resources.

On the other hand, the network load is initially higher, as is the case with the proactive FRR mechanisms. This is the result of the initial EM-MREP flooding process. However, the EM-REP was not designed to protect all flows affected by a failure. EM-REP protects only specific but important customer data flows that require special treatment or lossless delivery through an ISP. These flows are only a subset of all flows affected by the error. In addition, we expect that the EM-REP mechanism, like other IPFRRs, will only work for a short time, not longer than a few tens of milliseconds or a few seconds. It ends when the network convergence process is complete, the multicast tree is no longer used, and packets of protected flows are routed again as unicast packets.

#### **7. Conclusions**

The paper presents the Enhanced Multicast Repair (EM-REP) FRR mechanism, which solves several limitations of the legacy M-REP FRR mechanism. This means mainly support for fast reroute in the event of continuous link and node failures throughout the whole network and that the destination host does not have to be directly connected to a network with a router that performs decapsulation, which also reduces a flooding process of M-REP packets in a network with multiple routing areas.

Both mechanisms belong to the family of Fast Reroute solutions. The EM-REP mechanism presented in this paper, makes it possible to create an alternate backup path that allows packets to bypass the failures of one or more links or nodes at a given time. To achieve this goal, the EM-REP has been built on two cornerstones, the PIM-DM protocol and tunneling. The PIM-DM delivers multicast data thorough a distribution tree constructed by the flooding and reverse pruning. The EM-REP is based on this behavior, where in the event of a failure (or even multiple failures), for specific traffic flow an alternate path is built by PIM-DM flooding and pruning. In this case, the router begins encapsulating unicast packets of the protected flow into multicast packets flooded out and around the failure. To ensure correct operation of routers, we have not modified the PIM-DM process as such, i.e., for a common multicast traffic the PIM-DM process works as usual. However, for the correct construction of alternative FRR paths (distribution trees), we have modified the PIM-DM RPF process, where we use the rule of the first arrival. Alternative paths are created only for protected flows, so the router must identify correct packets in some way. In short, we expect that flow identifiers are predefined and preconfigured by the network administrator in advance. However, in future, some dynamic distribution mechanism may be used, inspired, for example, by those used for a dynamic distribution of Rendezvous Point (RP) addresses (Auto-RP, BSR Bootstrap Router mechanism), but here used for the distribution of protected flow identifiers.

As has been already mentioned, most FRR mechanisms require the pre-calculation of alternative routes for different network failure scenarios. On the one hand, these preparatory calculations have undesirable effects on the router's limited resources, such as CPU load and memory. On the other hand, they may depend on a specific link-state routing protocol. The EM-REP mechanism does not require any preparatory calculations, which is effective for IoT devices such as sensors.

Moreover, the EM-REP does not depend on any unicast routing protocol. In addition, although EM-REP can bring benefits resulting from the use of a specific routing protocol supporting the organization of unicast routing to areas, as is presented here for OSPF, it could work for IS-IS as well. Furthermore, the EM-REP FRR mechanism provides 100% repair coverage for single as well as multiple failures occurring at different times and places in the network. Finally, EM-REP eliminates the condition of directly connected destination to router D of legacy M-REP.

The EM-REP mechanism uses the generic flooding process of the PIM-DM protocol to provide the protection for specific flows that expect special handling inside the network. The behavior and goal are generic enough with a wider application domain. In the area of WSN and the IoT, it can be used to distribute, for example, urgent messages across the WSN network or to assure the time-critical delivery of important information from sensors to gateways or behind to analytic servers.

The EM-REP was fully implemented, and its correctness was tested using the OMNeT++ simulator. We have performed extensive tests of the implementation in different networking scenarios, which validated the functional correctness of all the mechanism functions. The principle of the mechanism is unique, and it is possible to apply it in other networks such as WSN, IoT architecture, and other areas as well, which will be studied in future work.

**Author Contributions:** Conceptualization, J.P. and P.S.; software, J.P.; validation, J.P. and P.S.; formal analysis, J.P., P.S., O.Y., I.B., M.H.; investigation, J.P.; resources, J.P.; data curation, J.P.; writing—original draft preparation, J.P., P.S., O.Y.; writing—review and editing, J.P., P.S., O.Y., I.B., M.H.; investigation, J.P.; visualization, J.P.; supervision, J.P. and P.S.; project administration, J.P. and P.S.; funding acquisition, J.P. and I.B.. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by UNIZA Grant System and Faculty Research Grant (FVG).

**Acknowledgments:** This work has been supported by the Slovak Scientific Grant Agency (VEGA) grant agency, Project No. 1/0626/19 "Research of mobile objects localization in IoT environment".

**Conflicts of Interest:** The authors declare no conflict of interest.
