Evaluation of Traffic Engineering Routing Models Based on Type of Service in Communication Networks

Abstract

This article is devoted to enhancing the mathematical models for traffic engineering routing and load balancing to ensure differentiated quality of service for packet flows with varying priorities in communication networks. The research focuses on formulating updated load-balancing conditions considering packet-flow priorities and conducting a comparative analysis of different models to identify the most effective solution. The study introduces a multipath routing model based on the type of service and presents numerical research to evaluate the proposed solutions in varying network conditions. The findings allowed us to select the model that offers the most precise regulation of service differentiation for different priority flows by optimizing the load-balancing strategy. This model directs higher-priority packet flows through less heavily loaded links and routes, while lower-priority flows are sent through more heavily loaded paths. Moreover, the routes for higher-priority flows are designed to have fewer hops than those for lower-priority flows. This approach improves network resource utilization and simplifies the architecture for delivering differentiated quality of service, potentially reducing the need for the manual configuration of congestion management mechanisms. The study concludes that the proposed enhanced model is the most effective for practical implementation in modern networks, offering a robust solution for managing diverse priority traffic.

1. Introduction

The primary objective of modern communication networks remains the assurance of quality of service (QoS) [1,2,3]. The significant expansion of services that these networks offer, as well as heightened QoS requirements, necessitate continuous advancements in communication technologies and traffic management protocols [1,4]. Network performance and the associated metrics are critical in ensuring end-to-end user service quality. Therefore, network service providers must meet performance standards, utilizing network resources efficiently and reliably without incurring losses. Key performance indicators typically include delay, jitter, packet loss, and bandwidth [1,5]. It is also essential to consider that different types of traffic have varying requirements for these indicators; for example, multimedia streams demand minimal delay and jitter, while data traffic is more sensitive to packet loss. Consequently, in communication networks, differentiated QoS (DiffServ) [6] is achieved through an integrated approach employing various complementary technological tools, including routing with load balancing.

It is well established that network link utilization, router interface queues, and the efficiency of network routes influence key QoS indicators. Therefore, ensuring a balanced use of network resources is vital. Inefficient allocation of these resources can lead to the overloading of specific devices and network segments while other areas remain underutilized, ultimately resulting in a decline in QoS level.

Traffic engineering (TE) is crucial in managing packet flows within network infrastructures, significantly enhancing network performance by regulating traffic and alleviating congestion [7,8]. Effective TE mechanisms can minimize network downtime by preventing the disruptions caused by bottlenecks, thereby ensuring network availability and reducing the likelihood of costly node and link failures. TE also enhances user QoS by delivering the necessary performance metrics. Additionally, TE optimizes resource utilization, reducing the need for excessive redundancy and, thus, lowering costs. As networks grow, TE provides scalability by dynamically adjusting packet flow paths to accommodate changing demands without compromising performance.

The principles of traffic engineering [7,8,9,10,11,12,13,14,15,16,17,18] are widely employed to address the network challenges related to routing and load balancing. This widespread adoption is primarily due to TE’s ability to optimize existing network resources without requiring additional investments. Consequently, advancing the mathematical models and methods of TE routing, considering QoS requirements and traffic adaptation, remains a critical scientific and practical task.

Therefore, this work aims to enhance the mathematical model for TE routing and load balancing to ensure differentiated quality of service for packet flows with varying priorities. The main objectives include formulating updated load-balancing conditions that account for packet-flow priorities, conducting a comparative analysis of the effectiveness of different load-balancing conditions, and selecting the most effective model. Additionally, the study provides recommendations for the practical application of this model in modern communication networks. The tasks of this research are defined as:

• Analyze existing traffic engineering (TE) principles and technologies by comprehensively reviewing current TE specifications, technologies with TE support, and advanced solutions.
• Develop an updated multipath routing model based on type of service (ToS) by formulating new load-balancing conditions, prioritizing packet flows according to their QoS requirements, and defining a mathematical model integrating ToS into traffic engineering for more precise service differentiation.
• Perform a numerical analysis of the proposed TE solutions by conducting numerical experiments on different network topologies using the proposed ToS-based routing models and analyzing two variants of initial data to assess the performance of different load-balancing conditions.
• Evaluate the effectiveness of load-balancing models in handling packet flows with different priorities and compare the results to identify the most suitable model for achieving a differentiated QoS.
• Discuss the implications of the numerical research results regarding their practical application in communication networks.
• Provide recommendations for implementing the most effective load-balancing model in real-world networks to improve service differentiation, based on flow priority.
• Conclude the study by summarizing the research findings, thereby highlighting the significance of the selected model and its contribution to enhancing the QoS in traffic engineering.

The remainder of this study is structured as follows. Section 2 provides an overview of traffic engineering specifications, existing TE-supporting technologies, and a related work analysis. Section 3 defines the traffic engineering multipath routing models based on the type of service (ToS) in communication networks. Section 4 presents a numerical analysis of network topologies using the proposed ToS-based traffic engineering solutions, evaluating and comparing the effectiveness of various load-balancing models to determine the most suitable one. Section 5 discusses the research findings and offers recommendations for applying the best load-balancing model to achieve precise service differentiation for different priority flows. Finally, Section 6 concludes the study.

2. Overview of Technological and Theoretical Solutions for Traffic Engineering

2.1. Analysis of the Basic Principles of Traffic Engineering in IP Networks under Existing Specifications

The specification given in Ref. [7] describes the general principles of traffic engineering in IP networks, focuses on TE within a particular domain (e.g., an autonomous system, AS), and also considers inter-domain TE. The document generally aims to specify the best practices for TE in IP networks.

Thus, Ref. [7] defines the major network function as routing and forwarding traffic between the respective nodes. At the same time, one of the most characteristic functions performed by traffic engineering is controlling and optimizing the routing and forwarding processes. Finally, the primary purpose of traffic engineering is to solve the problems of performance evaluation and network optimization and to apply leading technologies and scientific principles to evaluate, describe, model, and manage traffic.

When implementing traffic engineering in practice, it is, therefore, necessary to effectively respond to various network events, such as network congestion (actual or predicted), link or node failures, degradation of the quality of service, changes in traffic or network structure, etc. It should also be borne in mind that optimization approaches can be proactive or reactive. Proactive traffic management occurs if preventive measures are taken to protect against predicted adverse network conditions. Reactive management, based on the TE concept, allows you to adaptively respond to adverse events occurring in the network (congestion or failures). Another critical task of implementing TE is to ensure reliable network operation by implementing mechanisms to improve its integrity and survivability. This reduces the negative impact of errors, malfunctions, and failures in the network on user service [7].

It is also worth noting the effectiveness of the optimization approach for managing bandwidth and traffic. Bandwidth management is usually understood as planning, controlling routing processes, and managing network resources, including link bandwidth, buffer space, and network devices (routers, switches, and controllers) or other computing resources. At the same time, traffic management is associated with the corresponding functions implemented on nodes (queue management, scheduling, etc.), regulating and distributing packet flows in the network and allocating network resources to them.

The current TE also aims to implement automated control capabilities adapted to network state changes while maintaining reliable networking [3]. In addition, TE mechanisms must meet existing requirements but be flexible and extensible to meet unpredictable future needs and changes [2,3,4,9].

Since traffic engineering ensures the optimal and reliable use of network resources, it takes place at the controller and data plane levels in software-defined network architectures. At the same time, TE solutions include those that contain the following key elements [7]:

• path selection policy;
• path steering, as implemented in RSVP-TE (TE extensions to the resource reservation protocol) [10], segment routing (SR) [11], etc.;
• resource management (redundancy, bandwidth allocation, network device queue management, etc.) to control losses and delays.

The general classification of traffic engineering systems can be presented as shown in Figure 1 [7]. Thus, the TE system is understood as a set of objects, mechanisms, and the protocols used to achieve traffic engineering goals.

The classification of TE systems described in Ref. [11] includes different styles and types. Thus, TE systems can be time-dependent, making decisions based on the dynamics of traffic changes, for example, during peak hours; state-dependent, adapting to the current network state, for example, regarding link congestion or available bandwidth; or event-dependent, responding to specific events, for example, link failures. Conversely, TE systems can operate offline, for instance, with predictive routing plans that require significant computational overhead but do not require real-time decision-making, or online, with dynamic, simple real-time adjustments.

Centralized traffic engineering involves decision-making by a central (controlling) node, such as an SDN controller, while distributed traffic engineering uses the distribution of decision-making among the network elements. In practice, however, most TE systems are hybrid, containing both centralized and distributed functionality. Thus, TE can use local information about the network state, applying a decentralized management strategy or global information, which is a prerequisite for centralized management.

Prescriptive traffic engineering recommends specific actions, while descriptive TE provides insight into the network state without suggesting specific actions and aims to evaluate the impact of various policies. Open-loop management within a TE operates without feedback, relying on predefined rules, while closed-loop TE involves continuous monitoring and feedback to adjust decisions. Finally, TE can be tactical, focusing on short-term adjustments, or strategic, involving the long-term planning of specific management policies.

Figure 2 shows the basic mechanisms and related RFC specifications when implementing the traffic engineering concept according to Ref. [7].

Also, among the techniques used by TE mechanisms, the IETF defines the following [7]:

• Constraint-based routing;
• RSVP and RSVP-TE;
• MPLS and generalized MPLS (GMPLS);
• IP performance metrics (IPPM);
• Flow measurement;
• Endpoint congestion management;
• TE extensions to the IGPs (IS-IS and OSPF)
• BGP—link state;
• Path computation element;
• Segment routing (SR);
• Tree engineering for the bit index’s explicit replication;
• Network TE state definition and presentation;
• System management and control interfaces.

2.2. Technologies with TE Support

Technological solutions with TE support usually address the issue of ensuring the balanced use of the available network resource by implementing a multipath routing strategy when packet flows between the source and destination nodes are transmitted via multiple paths. At the same time, any solution should describe the method of selecting or calculating routes, their number and characteristics, and the ability to balance flows with them. Here are the most well-known technological solutions for TE routing.

MPLS traffic engineering (MPLS TE) enables network operators to efficiently manage traffic flows in the MPLS core by utilizing the available bandwidth between routers more effectively than traditional IP routing and using interior gateway protocols that rely on static metrics and shortest-path calculations [8,9]. MPLS TE uses constraint-based routing to determine the optimal path according to criteria such as bandwidth, interfaces, or specific routers, creating unidirectional LSP tunnels with defined requirements. The main components of MPLS TE are the link information distribution mechanism, which notifies the user of additional link attributes; an advanced shortest path algorithm with CSPF constraints; path configuration with RSVP-TE [10,12], a resource reservation protocol; and LSP creation, including route configuration, maintenance, deletion, and error signaling. Traffic is also transported through TE tunnels using static routes, policy-based routing, or MPLS TE-specific methods.

Generalized multi-protocol label switching (GMPLS) extends MPLS capabilities by integrating different switching technologies, thereby optimizing the use of network resources [13]. GMPLS is based on three main protocols: RSVP-TE, which manages signaling [12]; the extended OSPF-TE protocol, which provides routing information for TE paths [14,15]; and the LMP link management protocol, which controls link connectivity and fault management. It should be noted that OSPF-TE plays a crucial role in information dissemination in GMPLS technology.

IS-IS-TE with traffic engineering extension enhances the IS-IS routing protocol by disseminating network performance information, such as that on delay and available bandwidth, to improve path selection decisions [16,17,18]. This protocol offers a scalable method of disseminating performance metrics in MPLS/GMPLS networks, making it particularly valuable when performance criteria such as latency are critical to data path selection.

The TE principles have also been implemented in fault-tolerant routing protocols, namely, the first hop redundancy protocols (FHRPs) [19,20,21,22]. For example, the GLBP (gateway load balancing protocol) [20,21,22] supports several modes of load balancing between border routers that create a virtual default gateway. Such solutions increase the network reliability and resiliency at the access level and reduce packet loss in the event of gateway failures by implementing round-robin or weighted load balancing. Improvements have also been extended to exterior gateway protocols, such as the BGP (Border Gateway Protocol) [23,24], in conjunction with segment routing (SR) [25].

2.3. Advanced Solutions with TE Support

The analysis of advanced solutions with TE support has previously been conducted [26,27,28,29,30,31,32,33,34,35,36,37,38,39,40].

Table 1. Current traffic engineering routing solutions overview.

Ref.	Year	Key Contribution	Underlying Approach	Application
[26]	2023	Heuristic traffic engineering algorithm SR-ELS that effectively reduces maximum link utilization and improves traffic engineering in segment routing networks.	Heuristics	Segment routing network
[27]	2023	Router activation heuristics for energy-saving ECMP and valiant routing in data center networks effectively reduce energy consumption in computing systems; energy-aware routing.	Heuristics	Data center network
[28]	2019	Model-free traffic engineering framework that adopts multi-agent reinforcement learning for distributed control to minimize end-to-end delay in large-scale multi-hop networks.	Multi-agent learning system and reinforcement learning	Large-scale network
[29]	2023	Fuzzy-based load-balanced opportunistic routing for asynchronous duty-cycled WSNs (FLORA) that achieves better performance in terms of energy consumption, overhead packets, waiting times, packet delivery ratios, and network lifetime compared to other protocols.	Fuzzy logic	WSN
[30]	2023	A self-driving system for intelligent flow routing in programmable networks is proposed. Compared to equal-cost multi-pathing (ECMP), it improves load-sharing and path utilization in programmable networks.	Machine learning traffic prediction	Programmable networks
[31]	2021	The traffic engineering weight adjustment algorithm WASAR is used to optimize routing in a dynamic hybrid SR network.	Optimization	Hybrid segment routing networks
[32]	2023	The traffic engineering approach, combining contrastive learning and reinforcement learning, significantly improves hybrid SDN performance by adapting to fast-changing network flows.	Optimization, contrastive learning, and reinforcement learning	Hybrid SDN
[33]	2022	A destination-based traffic engineering solution, FlexEntry, reduces time complexity and routing update overhead while maintaining good network performance by intelligently selecting critical entries with reinforcement learning and optimizing traffic split ratios with linear programming.	Optimization and reinforcement learning	SDN
[34]	2023	A flexible and disturbance-aware traffic engineering solution, FlexDATE, which can achieve near-optimal performance, can generalize well to unseen traffic scenarios, and will remain resilient to single-link failures.	Optimization, reinforcement learning, and linear programming	SDN
[35]	2023	A scalable learning-based TE, Roracle, which can quickly predict a good routing strategy for a long sequence of future traffic matrices.	Optimization, supervised learning, and linear programming	SDN
[36]	2021	The RACKE+AD system, which combines oblivious routing and average delay, significantly improves the performance and resource utilization of software-defined networks.	Optimization	SDN
[37]	2023	MOLS is a new segment routing-based optimization algorithm that performs similarly to conventional methods but requires fewer policies. This results in faster deployment and the removal of congestion in sub-second time frames.	Midpoint optimization	Backbone networks
[38]	2023	QoS-Aware adaptation traffic engineering solution for multipath routing when the network load balancing is optimized, so that more priority flows are routed through links that are less loaded than those links through which packets of lower-priority flows are transmitted.	Optimization	SDN
[39]	2023	Two modifications of the traffic engineering routing were created, including the linear limitation model (TER-LLM) and traffic engineering limitation (TER-TEL), each considering the main features of packet flow: intensity and priority.	Optimization	Network Edge
[40]	2022	Secure traffic engineering routing model with modified load-balancing conditions, considering network characteristics such as topology, features of the traffic being transmitted, and the link bandwidth and the probabilities of their being compromised. The model allows for the reduction of the links load with a high value of compromise probability, while more traffic will be transmitted over secure links.	Optimization	SDN

provides an overview of current traffic engineering routing solutions, highlighting several key contributions from advanced TE research. The table illustrates the growing popularity of heuristic solutions, reinforcement learning (RL), and machine learning (ML) approaches in addressing TE challenges. It emphasizes the joint use of optimization techniques and ML to enhance the efficiency of TE strategies.

A significant portion of TE solutions is applied within software-defined networks (SDNs). Additionally, TE demonstrates its applicability in both backbone networks and at the network edge. SDN solutions leverage centralized control and programmability to optimize traffic flow and resource utilization dynamically.

Ultimately, advanced TE research is vital for improving the quality of service in modern networks. By integrating advanced techniques like ML and RL with traditional optimization methods, TE solutions can better adapt to changing network conditions and demands, ensuring reliable and efficient network performance. Furthermore, this analysis underscores the need to develop new multi-aspect models that consider not only the network and its link utilization but also the type of service (ToS) seen during traffic distribution. This complexity highlights the challenges in modern network management and the need for comprehensive solutions.

3. Traffic Engineering Multipath Routing Model, Based on the Type of Service

In this study, the mathematical model proposed in [38] will be utilized to derive traffic engineering solutions for multipath routing, based on the type of service (ToS-TE) found in communication networks. The primary reasons and benefits of adopting this approach include its optimization foundation, predictable computational complexity, and consideration of the topological and functional network parameters, as well as traffic characteristics.

To ensure clarity and a focus on load-balancing processes, this article will employ the model described in [38]. To achieve this, we have defined our notations in

Table 2. Notation summary.

Symbol	Meaning
$G=\left(R,E\right)$	graph presenting the network structure
$R=\left\{R_i;\hspace{0.33em}i=\overline{1,m}\right\}$	set of vertices simulating the routers
$E=\left\{E_{i,j};\hspace{0.33em}i,j=\overline{1,m};\hspace{0.33em}i\ne j\right\}$	set of arcs representing the links
$s_k$	source node
$d_k$	destination node
$K$	$\mathrm{set} \mathrm{of} \mathrm{flows} \mathrm{for} \mathrm{transmitting} \mathrm{in} \mathrm{the} \mathrm{network} (k\in K$)
$x_{i,j}^k$	$k\mathrm{th} \mathrm{flow} \mathrm{fraction} \mathrm{in} \mathrm{the} \mathrm{link} E_{i,j}\in E$
${\phi }_{i,j}$	$\mathrm{link} E_{i,j}\in E$ capacity (packets per second, or pps)
${\lambda }^k$	kth flow average packet rate (pps)
$p{r}^k$	kth flow precedence
${\rho }_{i,j}$	$\mathrm{link} E_{i,j}\in E$ utilization coefficient

.

To implement multipath routing with load balancing, the following constraints are applied to the routing variables:

$$0\le x_{i,j}^k\le 1, k\in K, E_{i,j}\in E.$$

(1)

The conditions for flow conservation in a network can be expressed as follows:

$$\left\{\begin{array}{l}{\displaystyle \sum _{j:E_{i,j}\in E}{x}_{i,j}^k}-{\displaystyle \sum _{j:E_{j,i}\in E}{x}_{j,i}^k}=0;k\in K{, \mathrm{R}}_i\ne s_k,d_k;\\ {\displaystyle \sum _{j:E_{i,j}\in E}{x}_{i,j}^k}-{\displaystyle \sum _{j:E_{j,i}\in E}{x}_{j,i}^k}=1;k\in K{, \mathrm{R}}_i=s_k;\\ {\displaystyle \sum _{j:E_{i,j}\in E}{x}_{i,j}^k}-{\displaystyle \sum _{j:E_{j,i}\in E}{x}_{j,i}^k}=-1;k\in K{, \mathrm{R}}_i=d_k.\end{array}\right.$$

(2)

Meeting condition (2) ensures that all packets of the kth flow are transmitted through the network without any loss. To maintain a balanced load and prevent network communication links from becoming overloaded, additional conditions are incorporated into the mathematical model:

$${\displaystyle \sum _{k\in K}{\lambda }^k{\mathrm{x}}_{i,j}^k}\le \alpha {\phi }_{i,j}, E_{i,j}\in E,$$

(3)

where the control variable $\alpha$ is the upper bound of the network link utilization:

$$0\le \alpha \le 1.$$

(4)

In Ref. [38], it was established that the variable $\alpha$ limits the maximum values of the utilization coefficient:

$${\rho }_{i,j}={\displaystyle \frac{{\displaystyle \sum _{k\in K}{\lambda }^k{x}_{i,j}^k}}{{\phi }_{i,j}}}\le \alpha , E_{i,j}\in E.$$

(5)

In previous works [41,42], the TE routing problem was formulated as an optimization task with the following objective function, while satisfying constraints (1)–(4):

$$\underset{x,\alpha }{\min } \alpha .$$

(6)

Here, the optimization problem is of the linear programming class. In a perfect scenario, all network links would maintain the same minimum utilization. However, achieving equal utilization across all links is practically impossible, due to real-world factors such as network topology, differences in the link capacities, and varying packet flow directions.

The novel aspect of the solutions proposed in this work is the modification of condition (3) to account for DiffServ-QoS requirements based on packet-flow priorities. Similar to the approach in Ref. [38], this modification ensures that, when implementing a multipath routing strategy, higher-priority packet flows are directed through links with lower utilization rates. This adjustment aims to improve QoS for higher-priority packet flows by reducing average packet delay and probable packet loss, which are directly influenced by link utilization. This can be achieved by adjusting the virtual intensity of flows according to their priority values in the load-balancing Equation (3).

In modern communication networks, packet priorities are typically indicated in the packet headers. For example, in IPv4, priority is specified using either three or six bits in the type of service (ToS) byte [7]. The IP precedence method is used in the former case, while the differentiated services code point (DSCP) method is used in the latter [6]. In IPv6, a similar field in the packet header is referred to as the traffic class byte. For MPLS technology, packet priority is indicated in the three-bit EXP field. In this study, solutions related to TE routing and load balancing based on packet priorities will be referred to as ToS-TE.

In Ref. [38], the ToS-TE solution was limited to applying only to disjoint routes in the network. In the present work, due to the flow conservation conditions (2), ToS-TE can be implemented on paths with an arbitrary intersectional or non-intersectional nature, significantly expanding the proposed model scope. For example, in [38], it was proposed to modify the load-balancing language (3) and present it in the next form:

$${\displaystyle \sum _{k\in K}(H(p{r}^k+1)+{\lambda }^k)x_{i,j}^k}\le {\alpha }_m{\phi }_{i,j}, E_{i,j}\in E,$$

(7)

where H is a weighting coefficient regulating the packet-flow priority’s impact on the link utilization; ${\alpha }_m$ is a virtual upper bound of link utilization, which can take a value greater than one, i.e.:

$$0\le {\alpha }_m.$$

(8)

The use of condition (7) indicates that the virtual increase in the flow intensity in the links is carried out according to the additive principle, in direct proportion to the value of flow precedence ($p{r}^k$).

In this work, for the following study and comparison, as an alternative to expressions (3) and (7), we introduce modified load-balancing conditions as inequalities:

$${\displaystyle \sum _{k\in K}H(p{r}^k+1){\lambda }^k{\mathrm{x}}_{i,j}^k}\le {\alpha }_m{\phi }_{i,j}, E_{i,j}\in E,$$

(9)

$${\displaystyle \sum _{k\in K}(H(p{r}_{norm}^k+1)+{\lambda }^k){\mathrm{x}}_{i,j}^k}\le {\alpha }_m{\phi }_{i,j}, E_{i,j}\in E,$$

(10)

$${\displaystyle \sum _{k\in K}H(p{r}_{norm}^k+1){\lambda }^k{\mathrm{x}}_{i,j}^k}\le {\alpha }_m{\phi }_{i,j}, E_{i,j}\in E,$$

(11)

where $p{r}_{norm}^k$ is a normalized value of flow precedence, defined as:

$$p{r}_{norm}^k={\displaystyle \frac{p{r}^k}{{\displaystyle \sum _{k\in K}p{r}^k}}}, k\in K.$$

(12)

Expressions (7) and (9) differ based on variants of packet flow intensity amplification, depending on their priority values. In expression (7), we model an additive gain when the flow intensity in the selected link is virtually increased by the value $H(p{r}^k+1)$. In expression (9), the amplification is already multiplicative since the intensity in the mathematical model is virtually increased by $H(p{r}^k+1)$ times.

Expression (10) is similar to (7), but in (10), the additive amplification of the flow intensity in the link is carried out in direct proportion to the value of the normalized flow priority $p{r}_{norm}^k$ (12), not the priority. The load-balancing conditions (11) and (9) combine the multiplicative nature of the virtual intensity gain in the network links, but in expression (11), as in (10), the gain is directly proportional to the normalized flow priority $p{r}_{norm}^k$ (12). It can be noted in advance that, according to expressions (12) $p{r}_{norm}^k\le p{r}^k$, therefore, in expressions (10) and (11), the virtual amplification of the intensity of the kth flow in the links is less intense than when using conditions (7) and (9), respectively. In this study, all four variants of the modified load-balancing conditions, (7) and (9)–(11), will be investigated and compared in terms of ensuring DiffServ-QoS.

Along with the load-balancing conditions (7) and (9)–(11), it is also advisable to use the conditions for preventing link overload:

$${\displaystyle \sum _{k\in K}{\lambda }^k{\mathrm{x}}_{i,j}^k}\le {\phi }_{i,j}, E_{i,j}\in E.$$

(13)

As shown in Ref. [38], the optimality criterion (6) should also be modified. In it, next to the virtual upper bound of link utilization, the sum of the ratios ${\displaystyle \frac{{10}^{p{r}^k}{x}_{i,j}^k}{{\phi }_{i,j}}}$ is introduced over all links:

$$\underset{x,\alpha }{\min }({\displaystyle \sum _{k\in K}{\displaystyle \sum _{E_{i,j}\in E}\frac{{10}^{p{r}^k}{x}_{i,j}^k}{{\phi }_{i,j}}+L_{\alpha }{\alpha }_m}}).$$

(14)

The weighting coefficient $L_{\alpha }$ in criterion (10) regulates the impact of the boundary value ${\alpha }_m$.

This modification seeks to achieve the following objectives:

• The additive form of the first term in Equation (14) emphasizes the selection of routes with the minimum number of hops.
• Including link capacity ${\phi }_{i,j}$ in the denominator prioritizes selecting links and routes with maximum capacity.
• Introducing packet-flow priority $p{r}^k$ as a power of 10 ensures that higher-priority flows are transmitted via routes with fewer hops than lower-priority flows, particularly when multiple routes possess identical bandwidths but differ in terms of hop counts.

4. Numerical Research of the Proposed ToS Traffic Engineering Solutions in a Communication Network

The current section outlines the numerical investigations carried out in the MATLAB R2020b environment to evaluate the effectiveness of the proposed ToS-TE solutions. The analysis is segmented into three parts, namely, examining the initial data and the research findings for the first and second variants of initial data. In this research, we focused on specific network topology to ensure clarity in illustrating the improvements and to conduct a detailed comparative analysis of the factors influencing model efficiency.

4.1. Analysis of Initial Data

In the course of the study, five traffic engineering solutions were compared:

• TE model, which describes a classical traffic engineering solution based on solving an optimization problem with an optimality criterion (6) and constraints (1)–(4);
• ToS-TE1 model, which is based on solving the optimization problem with an optimality criterion (14) and constraints (1), (2), (7), (8), and (13);
• ToS-TE2 model, which is based on solving the optimization problem with an optimality criterion (14) and constraints (1), (2), (8), (9), and (13);
• ToS-TE3 model, which is based on solving an optimization problem with an optimality criterion (14) and constraints (1), (2), (8), (10), and (13);
• ToS-TE4 model, which is based on solving an optimization problem with an optimality criterion (14) and constraints (1), (2), (8), (11), and (13).

Figure 3 illustrates the network topology utilized in the experiments. The network comprised five routers, interconnected by eight communication links. Five predefined paths were established between the first router, serving as the source for all packet flows, and the fifth router, designated as the destination:

• Path 1: $R_1\to R_2\to R_5$;
• Path 2: $R_1\to R_3\to R_5$;
• Path 3: $R_1\to R_4\to R_5$;
• Path 4: $R_1\to R_3\to R_2\to R_5$;
• Path 5: $R_1\to R_3\to R_4\to R_5$.

Paths 1, 2, and 3 did not intersect; they shared only the source and destination nodes. In contrast, Paths 4 and 5 intersected with each other and with Paths 1–3. The network topology was selected for its simplicity and clarity, as well as for its suitability to thoroughly investigate load balancing and quality of service indicators under conditions involving both intersecting and non-intersecting paths. Three packet flows, each with an intensity of 200 packets per second, were transmitted between the routers, with IP precedence values ranging from 0 to 7.

Table 3. Variants of the initial data for analysis.

Variant	Link Bandwidth, pps
	E1,2	E2,5	E1,3	E3,5	E1,4	E4,5	E3,2	E3,4
1	600	600	600	600	600	600	600	600
2	350	700	900	350	350	800	900	900

presents the initial data for two variants of link capacity, which also influenced the network route capacity. The first variant represents a scenario where all links and routes have equal bandwidth, specifically, 600 packets per second. The second variant is characterized by asymmetry in the bandwidth of both links and routes within the network.

The effectiveness of the solutions was evaluated using a system of indicators. These indicators included the utilization rates (5) and their maximum value, the upper bound of network link utilization (4). Additionally, the end-to-end delay (τ) was calculated for each path connecting the first and fifth routers. The end-to-end delay was defined as the sum of packet delays at the output interfaces of the routers along the path, with the operation of each interface being modeled, for example, by the commonly used M/M/1 packet service model.

4.2. Research Results for the First Variant of the Initial Data

Figure 4 presents the study results and comparative analysis of the TE and ToS-TE1 models for the first variant of the initial data (Table 3), where the IP priorities of the three packet flows were 7, 3, and 0, respectively. We analyzed the dynamics of changes in the end-to-end delay of packets from different flows and the upper bound of network link utilization as a function of the weighting coefficient H under load-balancing conditions. The TE model did not account for the packet priorities of different flows during load balancing, resulting in the same average delay for all flows (Figure 4a), which was 5 ms. Simultaneously, the upper bound of network link utilization (Figure 4b) was approximately 0.333.

The ToS-TE1 model provided load balancing while considering the packet priorities of different flows (Figure 4a). Higher packet-flow priority corresponded to a higher QoS level, indicated by lower end-to-end delays, confirming the adequacy of the obtained results. A distinctive feature of the ToS-TE1 model is its ability to adjust the differentiation level of the quality of service for packets of different flows by varying the parameter H. As the weighting coefficient H increases, the difference in end-to-end delay values for packets of different priority flows also increases. This adjustment is most effective when the parameter H ranges from 0 to 1000 (Figure 4a). Here, at higher values of H, the differentiation in delays stabilized and showed minimal change.

Using the ToS-TE1 model, compared to the TE model, slightly improved the quality of service for the packet flow with an IP precedence of 7, reducing the end-to-end delay from 5 ms to 4.13 ms (a 17.4% improvement). However, end-to-end delays increased for packets with lower precedence: for an IP precedence of 3, the delay increased to 7.85 ms (a 57% increase), and for an IP precedence of 0, the delay increased to 8.57 ms (a 71.5% increase). Additionally, the upper bound of network link utilization (Figure 4b) gradually increased when using the ToS-TE1 model from 0.333 to 0.61, representing an 83% increase. This increase can be considered a conditional cost for ensuring flow differentiation through load balancing.

Figure 5 presents the study results and comparative analysis of the TE and ToS-TE2 models for the initial data first variant (Table 3), where the IP priorities of the three packet flows were set at 7, 3, and 0, respectively. Compared to the ToS-TE1 model (Figure 4), the results in Figure 5 demonstrate invariance to the values of the weighting coefficient H. Specifically, for the ToS-TE2 model, regardless of the H value, the performance indicators for routing and load-balancing decisions immediately reached their stationary values (Figure 5).

In Figure 5, the end-to-end packet delay values for the different flows and the upper bound of network link utilization were identical to those in Figure 4 at the maximum values of the H coefficient (H > 1000). Therefore, the ToS-TE2 model did not permit adjustment of the level of QoS differentiation by changing the H parameter, a feature available in the ToS-TE1 model. However, there is no need for an additional adjustment of this weighting coefficient for new input data related to changes in network topology, flow characteristics, etc.

Figure 6 presents the study results and comparative analysis of the TE and ToS-TE3 models for the first variant of the initial data (Table 3), where the IP priorities of the three packet flows were 7, 3, and 0, respectively. The solutions for the ToS-TE3 model, shown in Figure 6, are generally similar to the results of the ToS-TE1 model (Figure 4). They are also sensitive to the values of the H coefficient, especially when its values are relatively low (H < 1000), which allows for the adjustment of the QoS level differentiation for flow packets.

However, the ToS-TE3 model provides differentiation at a slightly different level (Figure 6a). Compared to the TE model, the ToS-TE3 model improved the quality of service for the flow of packets with an IP precedence of 7, reducing the end-to-end delay from 5 ms to 4.6 ms (down to 8%). End-to-end delays increased for packets with lower precedence, with an IP precedence of 3 to 5.1 ms (up to 2%) and IP precedence of 0 to 5.74 ms (up to 14.8%). Ensuring QoS differentiation led to a gradual increase in the upper bound of network link utilization (Figure 6b) from 0.333 to 0.42, i.e., up to 26%.

The use of the ToS-TE4 model for the first variant of the initial data (Table 3) produced the results shown in Figure 7. The nature of the dependencies resembled those obtained for ToS-TE2. Specifically, the values of the end-to-end delay of packets for different flows and the upper bound of the network link utilization (Figure 6) corresponded to the stationary values of these indicators obtained for ToS-TE3 and were independent of H.

Table 4. Comparative analysis of the load-balancing models for the first variant of the initial data.

Model Type	End-to-End Delay for Flow IP-Precedence (Milliseconds)			Upper Bound of Link Utilization, α
	Flow 1	Flow 2	Flow 3
	IP Precedence 7	IP Precedence 3	IP Precedence 0
ToS-TE1	Down to 4.13 ms	Up to 7.85 ms	Up to 8.57 ms	Up to 0.61
	↓ to 17.4%	↑ to 57%	↑ to 71.5%	↑ to 83%
ToS-TE2	4.13 ms	7.85 ms	8.57 ms	0.61
	↓ in 17.4%	↑ in 57%	↑ in 71.5%	↑ in 83%
ToS-TE3	Down to 4.6 ms	Up to 5.1 ms	Up to 5.74 ms	Up to 0.42
	↓ to 8%	↑ to 2%	↑ to 14.8%	↑ to 26%
ToS-TE4	4.6 ms	5.1 ms	5.74 ms	0.42
	↓ in 8%	↑ in 2%	↑ in 14.8%	↑ in 26%

summarizes the performance indicators of the solutions obtained for the ToS-TE class models (from the first to the fourth) for the initial data’s first variant (Table 3), compared to a solution typical of the classical TE model.

4.3. Research Results for the Second Variant of the Initial Data

In addition, the second variant of the initial data (Table 3) was studied, wherein the links and routes in the network had significantly different bandwidths. This study included comparing the performance indicators for all five routing and load-balancing models. Figure 8 presents the study results and comparative analysis of the TE and ToS-TE1 models for the second variant of the initial data (Table 3). The IP priorities of the three packet flows remained unchanged at 7, 3, and 0, respectively. In this scenario, the TE model provided packet service for all flows with the same average delay of 6 ms (Figure 8a), and the upper bound of network link utilization was 0.375 (Figure 8b). These values served as benchmarks to compare the performance of the other four ToS-TE models.

The ToS-TE1 model facilitated load balancing while considering the priorities of packets from different flows (Figure 8a). Packets with the highest priority experienced a shorter end-to-end delay. Similar to the previous case, the ToS-TE1 model allowed for adjusting the level of QoS differentiation among packets from different flows by altering the coefficient H. Compared to the TE model, the ToS-TE1 model improved the QoS for the packet flow with an IP precedence of 7, thereby reducing the end-to-end delay from 6 ms to 5 ms (down to 17%). However, end-to-end delays for packets with lower precedence increased: for an IP precedence of 3, the delay rose to 12.7 ms (up to 2.12 times), and for an IP precedence of 0, it increased to 20.2 ms (up to 3.37 times). Using the ToS-TE1 model (Figure 8b) resulted in a gradual network imbalance and increased the upper bound of network link utilization from 0.375 to 0.835, i.e., up to 2.27 times.

Figure 9 presents the study results and comparative analysis of the TE and ToS-TE2 models for the second variant of the initial data (Table 3). Unlike the ToS-TE1 model (Figure 8), the solutions shown in Figure 9 are independent of the weighting coefficient H, with network performance indicators reaching their steady-state values immediately.

These values of end-to-end delay for the different packet flows and the upper bound of network link utilization (Figure 9) correspond to the maximum level observed in Figure 8 at a high coefficient of H (H > 1000). This confirms that the ToS-TE2 model, unlike the ToS-TE1 model, does not permit adjusting the level of QoS differentiation by changing the H parameter.

By analogy with Figure 6, Figure 10 presents the study results and a comparative analysis of the TE and ToS-TE3 models for the second variant of the initial data (Table 3). The solutions of ToS-TE3 (Figure 10) are generally similar to the results presented for ToS-TE1 (Figure 8. They are also sensitive to the values of the H coefficient, particularly when H < 1000, which provides control over QoS level differentiation for packets of flows with different priorities. However, the ToS-TE3 model provides differentiation, such that, compared to the TE model, the quality of service for a packet flow with an IP precedence of 7 is improved: the end-to-end delay decreased from 6 ms to 5–5.2 ms (a reduction from 13.7% to 17%). For packets with an IP precedence of 3, the end-to-end delay ranged from 5.56 ms to 6 ms (a reduction to 17.3%). For packets with an IP precedence of 0, the end-to-end delay ranged from 6.8 ms to 8 ms, representing an increase of 13% to 33%. Ensuring the QoS differentiation led to a gradual increase in the upper bound of network link utilization (Figure 10b) from 0.375 to 0.5, an increase of 33%.

Using the ToS-TE4 model for the second variant of the initial data (Table 3) produced the results shown in Figure 11. The dependencies observed were similar to those obtained for ToS-TE2 (Figure 9): the end-to-end delay values for packets of different flows and the upper bound of network link utilization (Figure 11) corresponded to the stationary values of these indicators obtained for ToS-TE3 (Figure 10). In other words, they were independent of the H coefficient.

Table 5. Comparative analysis of load-balancing models for the second variant of the initial data.

Model Type	End-to-End Delay for Flow IP-Precedence (Milliseconds)			Upper Bound of Link Utilization, α
	Flow 1	Flow 2	Flow 3
	IP Precedence 7	IP Precedence 3	IP Precedence 0
ToS-TE1	Down to 5 ms	Up to 12.7 ms	Up to 20.2 ms	Up to 0.835
	↓ to 17%	↑ to 2.12 times	↑ to 3.37 times	↑ to 2.27 times
ToS-TE2	5 ms	12.7 ms	20.2 ms	0.835
	↓ in 17%	↑ by 2.12 times	↑ by 3.37 times	↑ by 2.27 times
ToS-TE3	5–5.2 ms	5.56–6 ms	6.8–8 ms	0.375–0.5
	↓ From 13.7 to 17%	↓ to 17.3%	↑ From 13 to 33%	↑ to 33%
ToS-TE4	5 ms	6 ms	8 ms	0.5
	↓ in 17%	not changed	↑ in 33%	↑ in 33%

compares the performance indicators for the solutions of the second variant of the initial data (Table 3) obtained for the ToS-TE class models (from the first to the fourth) with the classical TE model.

5. Discussion

The research results presented in the previous section necessitate a thorough discussion to assess the strengths and weaknesses of the ToS-TE-class mathematical models. The effectiveness of these models was evaluated based on the end-to-end packet delay for various flows and the upper bound of network link utilization. Figure 4, Figure 5, Figure 6, Figure 7, Figure 8, Figure 9, Figure 10 and Figure 11 illustrate these indicators, while Table 4 and Table 5 compile and present the quantitative comparisons between the models. A primary distinguishing feature of the ToS-TE class models (from the first to the fourth) is the specific form of load-balancing conditions applied within the network, namely, (7) and (9)–(11). This characteristic significantly influenced the diversity of the study’s outcomes.

The first significant finding from the comparison of ToS-TE class models is that the additive nature of the amplification packet flow intensity based on priority values, as implemented in conditions (7) and (10), enabled the effective regulation of the QoS level differentiation within the network. This is evidenced by the fact that, as the weighting coefficient H increased, as shown in Figure 4, Figure 6, Figure 8, and Figure 10, the difference in end-to-end delay among packets of different priority flows also increased. Moreover, the most effective adjustment occurred when the parameter H varied between 0 and 1000 (see, for example, Figure 4 and Figure 8). At higher values of H, the level of QoS differentiation in packet delays stabilized and ceased to change.

The observed pattern can be viewed as both an advantage and a potential drawback of the ToS-TE1 and ToS-TE3 models. On the one hand, the weighting coefficient H in conditions (7) and (10) can be experimentally adjusted to achieve the desired level of differentiation in the end-to-end delay of packets for different priority flows. On the other hand, determining the optimal value of H requires additional, typically automated configurations at the routing protocol level.

A key feature of the ToS-TE2 and ToS-TE4 models is implementing a multiplicative approach to amplifying packet flow intensity, based on priority values under balancing conditions (9) and (11). This characteristic results in a fixed level of QoS differentiation in the network, independent of the weighting coefficient H (as shown in Figure 5, Figure 7, Figure 9, and Figure 11). While this greatly simplifies the practical implementation of these models by advanced load-balancing routing protocols, it also eliminates the possibility of adjusting the level of QoS differentiation and flexibility offered by the ToS-TE1 and ToS-TE3 models.

Another comparative analysis observation is that the packet service differentiation level was significantly influenced by how the packet priorities were incorporated. It was found that directly using the values of IP packet priorities $p{r}^k$ in balancing conditions (7) and (9) led to a rather rough and sometimes even inadequate differentiation in the end-to-end delay of packets from different priority flows. For instance, when applying the second variant of the initial data (Table 3), which was characterized by links and routes with varying bandwidths, the TE model provided undifferentiated routing and load balancing, resulting in an end-to-end packet delay of 6 ms. The ToS-TE1 and ToS-TE2 models, which utilized IP packet priority values of $p{r}^k$ in balancing conditions (7) and (9), achieved a 17% decrease in end-to-end delay for the highest priority of packet flow (an IP precedence of 7) (Table 5). However, this improvement was accompanied by a significant degradation in the QoS levels for flows with lower-priority values—IP precedences of 3 and 0. The end-to-end delay for these two flows increased to 12.7 ms and 20.2 ms, respectively, representing increases of 2.12 and 3.37 times. Conversely, in the same scenario, the ToS-TE3 and ToS-TE4 models, which utilized normalized IP packet priority values $p{r}_{norm}^k$ in balancing conditions (10) and (11), resulted in end-to-end delay values for packets with an IP precedence of 3 and 0 of 5.56–6 ms and 6.8–8 ms, respectively. This represents a 17.3% decrease in end-to-end delay for the packet flow with an IP precedence of 3 and a slight increase of 33% in the end-to-end delay for the packet flow with an IP precedence of 0.

Overall, the use of normalized IP packet priority values in balancing conditions (10) and (11) enabled more efficient utilization of the available network resources. While the ToS-TE1 and ToS-TE2 models required an increase in the upper bound of network link utilization from 0.375 to 0.835 (an increase of 2.27 times) to achieve differentiation in packet service levels, the ToS-TE3 and ToS-TE4 models addressed the same issue when increasing α by only 33%, from 0.375 to 0.5 (Table 5). This adjustment positively impacted the QoS level and the end-to-end packet delay.

A similar, though less pronounced, situation was observed in the first variant of the initial data, where the links and routes had identical bandwidths (Figure 4 and Figure 5, Table 4). For instance, using the ToS-TE1 and ToS-TE2 models increased the upper bound of network link utilization from 0.333 to 0.61 (an 83% increase). In contrast, the use of normalized IP packet priority values in the ToS-TE3 and ToS-TE4 models (Figure 6 and Figure 7) provided a more appropriate differentiation of QoS levels for packets with different priorities, requiring only a 26% increase in α from 0.333 to 0.42 (Table 5).

The results of the comparative analysis suggest that the ToS-TE3 and ToS-TE4 models are more appropriate for achieving the adequate differentiation of QoS levels according to packet priorities. These models allow for effective differentiation through a controlled increase in network utilization α, demonstrating more efficient use of the available network resources than the ToS-TE1 and ToS-TE2 models.

The proposed models should be practically implemented at the routing and load-balancing protocol levels in modern and future networks. Given that the ToS-TE3 model is more versatile and can replicate the decisions of the ToS-TE4 model at high values of the weighting coefficient H, the ToS-TE3 model should be selected as the basis for protocol-level implementation among those studied in this work. When initializing the model, it is advisable to start with the maximum values of the weighting coefficient H under balancing condition (11). If a reduction in the level of QoS differentiation among different priority flows is required, adjusting (reducing) the value of H by analyzing changes in the network state is recommended.

6. Conclusions

Ensuring QoS in modern information and communication networks is a complex challenge. The increasing number of users and the emergence of new services with higher QoS demands necessitate continual improvements to the existing methods and the development of new tools to maintain QoS. Routing protocols play a crucial and influential role in ensuring QoS. They are responsible for calculating one or more optimal routes and organizing efficient load balancing within the network. As the analysis suggests, implementing the principles of the traffic engineering concept is a promising approach for achieving effective load balancing. Additionally, to maintain QoS within the IntServ and DiffServ architectures, congestion management tools must be used alongside routing and resource reservation protocols. Mechanisms such as PQ, CQ, WFQ, LLQ, and others based on distributing interface bandwidth among queues are essential for providing a differentiated or guaranteed QoS.

The solutions proposed in this work aim to provide a differentiated QoS, not through queue management mechanisms but through a carefully calibrated load-balancing strategy. In this approach, higher-priority packet flows are transmitted over less heavily loaded links and routes, while lower-priority flows are routed through more heavily loaded paths. Additionally, routes for high-priority flows should involve fewer hops compared to those for lower-priority flows. This strategy simplifies the architecture for delivering a differentiated QoS by potentially eliminating the need for congestion management mechanisms, many of which still require manual configuration by network administrators.

Traditionally, the effectiveness of a specific protocol solution largely depends on the type of mathematical model and method on which it is based. Accordingly, this study enhances the mathematical models of TE routing [38,39] to enable the routing and load balancing that account for IP packet priorities, as indicated in the type of service byte of the IP packet header. The model that is proposed in [41,42] and expressed through Equations (1)–(4) and (6) satisfies the core requirements of the TE concept for network load balancing, while Ref. [38] offers an approach to implementing ToS-TE in scenarios where routing is performed along disjoint paths.

In this work, we propose and analyze a set of four optimization models within the ToS-TE class, with each model based on one of the updated load-balancing conditions of (7) and (9)–(11). The effectiveness of these routing solutions was compared to the basic TE solution in (1)–(4) and in (6) by evaluating end-to-end packet delays of different priority flows, and the upper bound of network link utilization (4). The novelty of these solutions lies in their adaptability for routing along paths of any type, including the intersecting paths within a network. All the mathematical models studied and compared in this work reduce the technological challenge of TE routing to that of solving a linear programming problem with criterion (6) or (14).

In this study, we investigated diverse scenarios using two variants of the initial data, as outlined in Table 3. The first variant assumes a homogeneous network with equal bandwidth for all links and routes, while the second variant introduces asymmetry, wherein the link and route bandwidths vary. This allowed us to explore the impact of both symmetric and asymmetric network topologies on the performance of the proposed models.

According to the initial dataset (Table 3), the study results generally confirmed the effectiveness of all the proposed ToS-TE class models in providing a differentiated QoS for different priority flows through load balancing (Figure 4, Figure 5, Figure 6, Figure 7, Figure 8, Figure 9, Figure 10 and Figure 11). The experimental results demonstrated that packets with a higher IP precedence value achieved a higher QoS level, as measured by end-to-end delay.

Based on the results of the comparative analysis (Table 4 and Table 5), the ToS-TE3 model emerged as the most effective among the four models within the ToS-TE class. It demonstrated the most precise regulation of service differentiation for different priority flows by adjusting the H coefficient under the flow conservation conditions (10). This model also ensured a minimal and controlled increase in network utilization (α), indicating a more efficient use of the available network resources than the ToS-TE1 and ToS-TE2 models. The advantages of the ToS-TE3 model, as represented by expressions (1), (2), (8), (10), (13), and (14), are attributed to the use of an additive virtual increase in packet flow intensity under load-balancing conditions (10), following the normalized values of IP packet priorities $p{r}_{norm}^k$ (12).

The main directions for future research include expanding the metrics used to evaluate network performance by incorporating additional QoS metrics, such as jitter and packet loss, particularly in scenarios where the network operates in near-overload conditions. Additionally, further analysis will focus on not only optimality but also other key network properties, such as scalability, robustness, and resilience, including fault tolerance and cyber resilience, across various network topologies, encompassing more complex and large-scale configurations. Successful implementation of these research goals will likely require developing and applying advanced dynamic and nonlinear models, which may increase the computational complexity of the resulting network solutions.