3.1.5. Preparatory Calculations

The analyzed FRR mechanisms work on a principle which is based on the fast detection of link failure with a neighboring router and precalculated alternative routes (precomputing). The high complexity of these precalculations is a problem area [26].

The computational complexity of individual FRR mechanisms increases with the increasing number of routers in the network. These calculations must be repeated if there is a change in network topologies and they are typically performed on routers as specific low-priority processes when the router's Central Processing Unit (CPU) is idle [26]. Thus, the FRR mechanism calculations take up the valuable time and system resources of the router. Based on these facts, we conclude that one of the problem areas of FRR mechanisms are preparatory calculations. All existing FRR mechanisms work on this principle.

Based on the problems thus identified and their problem areas, this document proposes the EM-REP FRR mechanism, an improved version of the M-REP algorithm. The main improvements of the EM-REP proposal focus on the protection of important unicast flows in the event of subsequent and recurring failures occurring concurrently over time. This is a unique feature, as we identified that existing IPFRR mechanisms provide the single failure protection.

Our M-REP algorithms are not dependent on any of unicast routing protocols in general, but EM-REP is enhanced in a way that provides an advantage in specific deployment scenarios, where the area design is used (for example OSPF or IS-IS). Here, we propose the modification of the Area Border Router (ABR) router behavior. This modification adds flexibility to optimize packet delivery through an "on-the-go" decapsulation process, compared to the old M-REP approach where it must be performed on the last router in the delivery chain. This is the second M-REP algorithm enhancement introduced in the paper.

## **4. M-REP FRR Mechanism**

Based on the analysis and identified problem areas introduced in the previous section, a M-REP FRR mechanism has been proposed. The M-REP mechanism does not require precomputation of an alternative route, it is not dependent on any unicast routing protocol, it does not use a metric calculation of the alternative route, and, finally, it provides full repair coverage.

The M-REP FRR mechanism uses a multicast [65–67] routing protocol, Protocol Independent Multicast-Dense Mode (PIM-DM), as its basis. PIM-DM, at the beginning of multicast transmission, floods multicast packets to all PIM-enabled routers in a domain. We decided to use this flooding feature as the basic behavior of our M-REP algorithm. However, to fit PIM-DM to our purpose, we modified its Reverse Path Forwarding control mechanism.

In this section, the description of original M-REP mechanism is provided. In the next section, we will follow up with the presentation of its improved version, the Enhanced M-REP mechanism (EM-REP). EM-REP can create alternative route that allows recovery from the occurrence of multiple and even parallel failures.

#### *4.1. Description of the Original M-REP Mechanism*

To describe the M-REP mechanism, the role of routers in IPFRR is modified as follows:


The M-REP mechanism is designed to protect only specific important customer data flows, delivered over an Internet Service Provider (ISP) network from the source S to the destination host D (Figure 2a, mark 1, red path). A router that detects a connection failure (link or node) becomes the source router S (Figure 2a, mark 2). At the same time as the failure was detected, routers start the process of network convergence. In this moment, the source router S begins to encapsulate the original unicast packets of the protected flow into a specific M-REP multicast flow identified by (S, G) pair. These multicast packets are sent directly out (utilizing the flooding process of Protocol Independent Multicast-Dense Mode protocol) on all active PIM-DM enabled interfaces of router S (Figure 2b, mark 3, dashed green arrows). This starts the process of creating a multicast distribution tree by the PIM-DM. The result of this flooding represents an alternative route around the detected failure (Figure 2b, mark 4, bold green arrows). Router S continues to perform this process of encapsulation and flooding of the protected unicast flow until the process of network convergence is completed.

However, before the network completes the convergence and the new shortest paths are calculated, R routers may receive a multicast packets of M-REP flow even through interfaces do not match their current selection of the correct Reverse Path Forwarding (RPF) interfaces. This statement is conditioned by the fact that the current routing tables have not yet been updated with the new information resulting from a link failure.


**Figure 2.** Principle of the M-REP mechanism

Incorrect selection of the RPF interface would prevent routers, with the M-REP mechanism implemented, to accept and forward the multicast M-REP flow. For this reason, we created the following modification to the RPF interface selection:

For a router that is not suitable for the M-REP multicast (S, G) flow, let an RPF interface be the one that allows the first multicast packet of a suitable multicast (S, G) flow.

The original PIM-DM communication processing and sending mechanism, as well as legacy PIM-DM RPF selection, are not modified. The revised RPF check will only apply to the specific range of multicast group addresses reserved for the M-REP mechanism.

The term "first packet" used in the modified RPF rule (the rule of first-arrival) refers to a multicast packet, the processing of which leads to the creation of a new multicast table entry for a specific (S, G) pair. This principle is in accordance with the standard rule of the PIM-DM protocol that the first multicast packet of a specific (S, G) pair requires the creation of a new entry in the multicast routing table. This record does not exist before the arrival of the first multicast packet of a specific multicast (S, G) pair.

After selecting a RPF interface, previous routers forward multicast packets from their other interfaces and active PIM-DMs. Each router has exactly one RPF interface for a specific multicast M-REP (S, G) pair.

An alternative path created by the M-REP IPFRR mechanism is random because its formation is conditional on the arrival of M-REP multicast packets to individual routers. Consequently, the alternative path created by the M-REP mechanism is not the shortest possible path (Figure 3). However, other IPFRR mechanisms do not generally provide the shortest alternative paths [30,62].

**Figure 3.** The first arrival rule.

The destination D router may be a provider edge (PE) router or a router that is directly connected to the destination network of the original protected unicast data flow. When multicast M-REP packets arrive at the destination D router, these encapsulated protected flow packets must be restored (decapsulated) to their original format and then routed according to the unicast routing table.

The decapsulation process of the M-REP multicast flow is performed by the destination router D. This router should be directly connected to the destination network of the original unicast packets. To restore the flow back to its original unicast form correctly, destination router D must have the original unicast flow information (source and destination addresses). The M-REP mechanism currently uses a tunneling technique, which means encapsulation of IPv4 unicast communication in new multicast packets (Figure 4). This technique is one of many possible solutions of how we can preserve the original source and destination address of the packet. Another option is to use extension headers (for example in IPv6).


**Figure 4.** The M-REP encapsulation of the unicast communication.

RPF in PIM-DM is a mechanism to ensure that packets are received on a correct interface and to prevent the creation of the micro-loops. For the needs of the M-REP mechanism, the original RPF was modified following the rule of First-Arrival. The research question we faced was whether this modification could cause micro-loops. The verification was performed using the mathematical proof by a contradiction [56], which confirmed that the modification does not cause routing loops. However, the M-REP mechanism operates in topologies which must meet two conditions, the network topology must use point-to-point links only and the target of original protected flow must be directly connected to router D.

The M-REP mechanism will result in exactly one path created between routers S and D. Switching from the alternate M-REP path back to the original unicast path after network convergence is controlled by a dedicated timer. This timer is set to a value greater than the unicast routing protocol convergence time.

It is important to point out that the legacy multicast tree creation procedure used in the PIM-DM protocol remains unmodified. A router with an empty list of output interfaces (OIL) for a specific flow (S, G) logs out of the multicast distribution tree by sending a Prune message from its RPF interface. The Prune message will also be sent out on non-RPF interfaces that received packets fom (S, G) pair.

#### *4.2. M-REP State Machine*

For a logical representation of the M-REP mechanism, state diagrams are created for S router (Figure 5), as well as D and R routers (Figure 6).

**Figure 6.** The M-REP state diagram for D and R routers.

The M-REP process of the mentioned routers can move between the states described in the Table 3:


**Table 3.** States of the M-REP mechanism.

## **5. Enhancements for M-REP**

Although the M-REP mechanism represents a new approach to addressing the IPFRR, this mechanism contains some limitations, which we removed with the proposed extensions. In this section we present the results of further research on M-REP mechanism enhancements. We have primarily focused on the treatment of multiple failures and the enhancement of specific deployments, i.e., the Area Border Router extension.

## *5.1. Multiple Failures*

So far, we have dealt with the failure of a single link or router at a time. In critical situations, multiple failures may occur at the same time (Figure 7). This situation is an issue for the M-REP mechanism because it is not able to find an alternative route, although an alternative path exists. Nevertheless, it should be noted, that most of FRR mechanisms analyzed are not able to solve this situation of multiple failures at the given time. Their principles simply do not allow for the correction of multiple failures.

**Figure 7.** M-REP: reroute in the event of multiple failures.

To solve this problem, we propose an extension of the M-REP mechanism, which is called the Swap method. To explain its principle better, we can divide the method into three separate steps. We were inspired by the Multiprotocol Label Switching (MPLS) technology and its functions: Push, Swap and Pop. The step definitions are as follows:


The principle of the EM-REP mechanism in the event of a single failure is unchanged compared to the original design. If another failure occurs at a different time to the original failure, the method Swap shall be used.

This interpretation implies that the used M-REP multicast address (after the first failure at time t) will be in the event of a further failure (at time t + x, where x is the time difference between the first and second failure) replaced by another predefined multicast address. However, this behavior is not efficient and can be optimized further. This implies the following behavior.

A router that detects a new connection failure on already used M-REP backup path of a particular multicast flow will replace the multicast destination address of the existing M-REP header with another multicast address. That is, a router detecting a new failure on the original M-REP backup path becomes the next local repair point, the router S. This will force the router to start a new flooding process but using a different multicast address for the M-REP flow.

This behavior is shown in Figure 8. The primary path for delivering unicast packets from Source to Destination is through R1 → R2 → R5 → D (red arrows). Router R2, which has detected at time *t* the first link failure on the primary path, becomes the router S (Figure 8a, mark 1). Next, router S (R2) begins to encapsulate the protected unicast flow to a specific M-REP multicast flow identified by (S, G) pair, and initiates the flooding process in the topology (Figure 8a, mark 2). Routers that use PIM-DM with modified RPF control, receive multicast traffic (the first-arrival rule) and create an alternative M-REP pathway. The path goes through R2 (S) → R4 → D. Routers on interfaces that do not have receivers for the M-REP multicast flow or receive multicast traffic as the second ones are pruned using Prune messages.

**Figure 8.** The EM-REP principle at multiple failures at different times.

At time t + x, router R4 has detected a second network failure (Figure 8b, mark 3), which occurred on the link through which the alternative M-REP route (S, G) leads. R4 becomes the next source router S2. R4 router replaces the destination multicast address of the original (S, G) multicast flow with the new destination multicast address (S, G + 1). Next, the flooding process in the network starts again (Figure 8b, mark 4). As a result, the new alternative M-REP path is created for the multicast flow (S, G + 1). The path goes through R4 (S2) → R5 → D (Figure 8b, mark 5). The whole resulting alternative M-REP path will go through R2 (S) → R4 (S2) → R5 → D. The part between R2 (S) → R4 (S2) was constructed as the multicast distribution tree for the (S, G) flow. The second part between R4 (S2) → R5 →D was constructed for the multicast flow (S, G + 1).

In the proposed solution, it is necessary to deal only with failures that have already occurred on the alternative route created by the EM-REP mechanism. Failures that occur outside of the alternative path do not affect or interfere with the path created and should not be addressed.

#### *5.2. ABR Extension*

The PIM-DM protocol, which is used by the M-REP mechanism, assumes that all connected end stations are interested in receiving the multicast traffic. Therefore, the PIM-DM router delivers multicast packets simply by flooding the packets to all active PIM-DM neighboring routers. PIM-DM routers that do not have receivers for a given multicast or have received multicast packets on interfaces, which do not pass RPF control, will be pruned from the distribution tree.

These processes take place in the beginning of multicast broadcasting and periodically later on, so they cause an unnecessary network load [68]. As the EM-REP mechanism uses these PIM-DM processes (flood and prune), they are only carried out until the network convergence process is complete. Subsequently, the routing protocol then takes control of the router's routing logic. From this point of view, networks consisting of several administrative areas appear to be problematic. In this case, the multicast (S, G) flow flooded by router S will be delivered to all routers in all administrative areas (Figure 9).

**Figure 9.** The process of PIM-DM flooding in multi-area Open Shortest Path First (OSPF).

Here, we propose the modification of the M-REP behavior applied on border routers of administrative areas (ABR). If we consider a network with applied OSPF routing, the area boundary routers are called the Area Border Router or the Autonomous System Boundary Router (ASBR). In this case, if a failure occurs in a given area, the ABR/ASBR router will act as a decapsulating router instead of the original D router. It means, that the ABR/ASBR router will decapsulate a specific M-REP multicast (S, G) flow back to the original unicast communication. The ABR behavioral design in OSPF is shown in the diagram (Figure 10).

Let us explain this process using the topology shown in Figure 11. The source sends its packets to the destination. R01 detects a link failure to the next-hop router and begins to encapsulate the unicast flow on the M-REP specific multicast flow (S, G). In this case, the boundary routers (ABR/ASBR) are R02, R12 and R21. Using the modified behavior for the ABR/ASBR routers (Figure 10), the specific multicast M-REP flow will not pass to the other areas.

**Figure 10.** The Area Border Router (ABR) router behavior after the arrival of M-REP packet.

**Figure 11.** The flooding process with modified ABR.

This principle also removes the original M-REP mechanism design requirement, which assumes that the router D is directly connected to the destination. This solution requires that two different areas are connected over one link and only one failure has occurred there.

Routers R12 and R21 will behave according to Action no. 1 (Figure 10). This means that the M-REP multicast will not be forwarded to the next area. The R02 router, however, will behave according to the Action no. 2 (Figure 10), which causes the decapsulation of the multicast M-REP flow and its further delivery to the destination.

#### *5.3. Manual Configuration of Router D*

Another way to select a router that performs the decapsulation of the M-REP multicast flow back to unicast is to manually configure a router as the decapsulating router (router D). In practice, the network administrator would manually select and configure a router to perform the decapsulation process (Figure 12).

**Figure 12.** Manual configuration of Router D.

An example of the situation, in which router D has to be manually configured when the destination of the protected unicast flow is not in the domain where a failure occurred is presented in Figure 12 (Manual configuration of Router D). In this case, the administrator must manually configure the Provider Edge Output (PeO) for router function D.

### **6. Evaluation of the EM-REP Proposal**

The functionalities of the enhanced version of the M-REP algorithm (EM-REP) proposed in Sections 5.1 and 5.2 have been verified by simulations. The implementation of the algorithm itself together and its extensions, as well as the creation of testing scenarios, were performed in the OMNeT++ discrete event simulator. The implementation is based on modification of the Automated Network Simulation and Analysis (ANSA) [69] and INET Framework Objective Modular Network Testbed in C++ (OMNeT++) libraries [70]. The ANSA library implements the multicast technology and the INET provides OSPF routing functionalities.

The correct behavior of the enhanced M-REP algorithm functions proposed in the paper has been successfully tested using several scenarios. The scenarios simulate various types of failures for various topologies. In these scenarios, we focus on the correctness of the partial activities of the algorithm, as well as on the investigation of the correct delivery of packets belonging to the protected flow to its destination. In this section, we introduce one of the comprehensive testing scenarios. The topology used in the scenario is shown in Figure 13. As a unicast routing protocol, the OSPFv2 protocol in a multiarea deployment model has been used. The routing domain consists of five OSPF areas, 23 routers and three hosts. For testing purposes, we generate data flow originated from host H11 to the H42 receiver. This data represents a protected stream of user datagrams, the delivery of which is ensured by our algorithm. In the case of a stable and error-free network condition, the delivery of packets is following the shortest path selected by OSPF (in Figure 13 represented by red arrows). In this scenario, we simulate the occurrence of several (three) independent network failures (the improvement introduced in Section 5.1), that occurs inside of different OSPF areas. The purpose of the simulation is to observe how the algorithm will protect user data in the event of multiple network failures within three separated areas.

**Figure 13.** The OMNeT++ simulation topology.

The description of scenario is as follows. At the beginning of the simulation we wait 200 ms to complete the process of network convergence (i.e., the convergence of OSPF unicast routing), then at the time of 200 sims (simulation seconds) the H11 host starts generating data flow. The source is the H11 with IPv4 address 192.168.11.2, the destination is the host H42 with IPv4 address 192.168.66.2. At the time of 210 sims we simulate the first failure, and the R14 router is shut down. At 212 sims we simulate the second failure as a permanent connection failure between routers R13 and R16. Finally, at 215 sims we simulate another failure of router R05 (Table 4).


**Table 4.** Test description.
