Path Load Adaptive Migration for Routing and Bandwidth Allocation in Mobile-Aware Service Function Chain

Abstract

With network function virtualization (NFV) expanding from network center to edge, the service function chain (SFC) will gradually approach users to provide lower delay and higher-quality services. User mobility seriously affects the quality of service (QoS) provided by the mobile-aware SFC. Therefore, we must migrate the SFC to provide continuous services. In the user estimable movement scenario with a known mobile path and estimable arrival time, we establish the estimation model of user arrival time to obtain the estimated arrival time. Then, to reduce the time that the user is waiting for the migration completion, we propose a softer migration strategy migrating mobile-aware SFC before the user arrives at the corresponding access node. Moreover, for the problem of routing and bandwidth allocation (RBA), to reduce the migration failure rate, the paper proposes a path load adaptive routing and bandwidth allocation (PLARBA) algorithm adjusting the migration bandwidth according to the path load. The experimental results show that the proposed algorithm has significant advantages in reducing the user’s waiting time by more than 90%, decreasing migration failure rate by up to 75%, and improving QoS compared to the soft migration strategy and two RBA algorithms.

1. Introduction

With the development of network virtualization, researchers have proposed network function virtualization (NFV) technology. NFV technology is usually combined with cloud computing, fog computing [1], Internet of things [2], etc. to form a new network structure. Service function chains (SFCs) decouple network services with high resource consumption into a group of virtual network functions (VNFs) connected in a specific order [3,4]. VNFs can be isolated from each other instead of the traditional network functions realized by proprietary hardware to reduce the deployment of dedicated hardware and network operation cost. To provide service with lower delay and higher QoS, SFCs are closer to users geographically. User mobility significantly affects the QoS provided by SFCs and even causes service interruption. Therefore, service providers must migrate related SFCs to maintain service connectivity. The challenges of SFC migration include migrating which VNFs [5], migrating to which virtual servers [6,7], choosing which migration path [8,9], and how to allocate migration bandwidth. The SFC migration strategy not only determines the migration time and success rate, but also affects the user’s QoS. Therefore, with more and more mobile users, SFC migration has gradually become a new research direction.

In a mobile scenario where users move from one network node to another, they access their services anytime and anywhere using mobile devices. Existing research on migration in mobile networks considers random and deterministic user movement (i.e., the arrival time and destination node are random, and both are certain). The well-studied challenges include maintaining service continuity and minimizing objective functions. For SFC deployment and migration in mobile edge computing networks, Chen et al. [10] designed a redeployment algorithm based on soft migration, which improves user satisfaction to a certain extent. For SFC migration in a distributed mobile network, based on locator/identifier separation protocol and software defined networking technology, Taleb et al. [11] achieved the tradeoff between migration cost and user experience under the condition of service continuity and user mobility. For SFC migration/remapping in cloud–fog computing environments, Zhao et al. [12] proposed the two-step migration strategy employing pre-copy and post-copy strategies to improve the success rate of SFC mapping. The above papers optimized the objective functions (QoS/migration cost/migration success rate) in the network where user mobility leads to migration.

Since resource allocation of migration significantly affects the QoS, some studies are currently devoted to the RBA problem of virtual machine (VM)/VNF migration. By using the consolidated middlebox, where each VM hosting VNF provides a service for a network service class, Xia et al. [13] formulated the path selection and capacity allocation as integer programming problems to minimize the migration cost; then, experiments showed the effectiveness of the algorithm. Yang et al. [14] studied bandwidth allocation among multiple live VM migrations in edge clouds to maximize the average QoS while meeting the migration time constraints; the method obtained a better QoS and convergence speed. For live VM migration in cross-data-center networks, Ayoub et al. [15] provided the distance adaptive RBA algorithm for the VM migration request according to the path length to minimize resource usage, and the algorithm achieved efficient network utilization.

The user’s movement path is certain, and the time user arriving at the destination node is estimable, which is called the user estimable movement scenario in the paper. This scenario is common in real life, such as buses [16,17]. It is worthy of attention and research to maintain continuous service in mobile networks involving vehicles. However, the existing migration strategy studies do not consider the user estimable movement scenario. Therefore, we address this gap in this study. In addition, instead of the existing soft migration strategy [12] migrating after the user reaches the corresponding access node, we propose a softer migration strategy migrating in advance to reduce user waiting time. For softer migration utilizing the estimable movement characteristics, we establish the prepared chain in advance to reserve resources, then transfer data by pre-copy, and finally resume the prepared chain to provide services for users. Moreover, when solving routing and bandwidth allocation problems, whether the migration is successful has a significant impact on the users’ QoS. However, few studies aimed to reduce the failure rate of SFC migration, and they did not fully consider available bandwidth resources of the migration path. Hence, to reduce the migration failure rate and let more users get timely and continuous services, in this paper, we propose the PLARBA algorithm adjusting the migration bandwidth according to the path load.

The main contributions of the paper are summarized below.

• We consider the user estimable movement scenario with known mobile paths and estimable arrival time. There is no research on migration in this scenario.
• Utilizing the estimable movement characteristics, we propose a softer migration strategy migrating before the user reaches the corresponding access node to reduce the user waiting time.
• We propose the PLARBA algorithm adjusting the migration bandwidth according to the path load to reduce the migration failure rate.

The related work is introduced in Section 2. We describe the system model and formulate the problem in Section 3. The strategy and algorithm are described in Section 4. The results are analyzed in Section 5. Finally, the paper is concluded in Section 6.

2. Related Work

2.1. Service Continuity for Mobile User

When users change their location [18,19], services follow the mobile users to ensure they always access the service from the best node. Otherwise, the user’s service will be interrupted. Thus, to maintain service continuity and reduce data transmission time in the link, the related SFC should be redeployed, and data from the original server should be copied to the new server, which triggers the SFC migration.

On the basis of network congestion, Nasrin et al. [20] proposed a virtual machine migration strategy migrating the tasks from the original congested edge server to the edge server with rich resources to improve the user’s QoS. Ding et al. [21] proposed task offloading and service migration strategies to optimize energy consumption or latency of user equipment with different mobility types. Islam et al. [22] took user mobility and cloud server load into account to select the best cloud server for the mobile VM and optimize the VM migration effectiveness.

Although [10,11,12,19,20,21] studied the migration problems caused by user mobility, they only focused on user determined and random mobility scenarios. Existing research scenarios are not comprehensive, and regular movements are common. Hence, the migration issue in the scenario with regular movement is worthy of further study.

2.2. Live Migration

Live migration [13,14] can significantly shorten the service interruption time during migration. To avoid service disruption, researchers need to address both memory data migration and service continuity. Live migration, which is also soft migration, is the leading migration method. The key aspect of live migration is the step-by-step copy.

The soft migration strategy has three modes to migrate memory data according to the handover time during migration: pre-copy [15], post-copy [12], and hybrid-copy [23]. Due to the strong robustness of pre-copy, most studies on VNF migration in dynamic network scenarios adopted this mode to balance resource consumption and operation overhead [24,25,26]. Addya et al. [27] proposed a modified serial migration strategy to migrate multiple VMs in a staggered pre-copy phase, thus balancing migration time and downtime. Kherbache et al. [28] proposed a scalable migration scheduler, which parallelly and sequentially migrates multiple VMs according to the memory workload and network topology to achieve the least completion time.

Although [12,13,14,15,23,24,25,26,27,28] fully studied the soft migration algorithm, the soft migration still has the possibility of downtime, which can affect the QoS of users. It is essential to propose other migration strategies in specific scenarios.

3. System Model

In this section, we mainly describe the system model of the paper. Firstly, we discuss the problem solved by the proposed RBA algorithm. Secondly, the physical network model, the user estimable movement scenario, and the SFC migration request model are introduced. Lastly, the mathematical expression of this problem is given.

Table 1. Notations in the paper.

Function	Notation	Definition
Parameter	${{\displaystyle G}}^P=\left({{\displaystyle N}}^P,{{\displaystyle E}}^P\right)$	Physical network, composed of the set of physical nodes and the set of physical links.
	$S=\left\{{{\displaystyle s}}_1,{{\displaystyle s}}_2,\dots ,{{\displaystyle s}}_{\left|SF\right|}\right\}$	The set of SFCs, where $\left|SF\right|$ is the number of SFCs.
	${{\displaystyle G}}^{i,S}=\left({{\displaystyle N}}^{i,S},{{\displaystyle E}}^{i,S},{{\displaystyle A}}^{i,S}\right)$	One mobile-aware SFC ${{\displaystyle s}}_i$, composed of the set of VNFs, the set of links, and the set of access nodes.
	${{\displaystyle N}}_m^{i.S}=\left\{{{\displaystyle v}}_1,{{\displaystyle v}}_2,\dots ,{{\displaystyle v}}_{\left|SM\right|}\right\}$	The set of migrated VNFs for ${{\displaystyle s}}_i$, where $\left|SM\right|$ is the number of migrated VNFs.
	$\left({{\displaystyle M}}_s,{{\displaystyle M}}_d,D,G,Q,C\right)$	VNF migration request.
	$U=\left\{{{\displaystyle u}}_1,{{\displaystyle u}}_2,\dots ,{{\displaystyle u}}_{\left|MU\right|}\right\}$	The set of users, where $\left|MU\right|$ is the number of users.
	${{\displaystyle T}}_u,{{\displaystyle T}}_a,\sigma$	Mean of the user arrival time, actual arrival time, and standard deviation.
	$P\left({{\displaystyle v}}_z\right),S\left({{\displaystyle v}}_z\right),M\left({{\displaystyle v}}_z\right)$.	CPU, storage, and memory resources of ${{\displaystyle v}}_z$.
	$RP\left({{\displaystyle n}}_x\right),RS\left({{\displaystyle n}}_x\right),RM\left({{\displaystyle n}}_x\right)$	Remaining CPU, storage, and memory resources of physical node ${{\displaystyle n}}_x$.
	$b\left({{\displaystyle l}}_j\right),RB\left({{\displaystyle e}}_i\right),d\left({{\displaystyle e}}_{{{\displaystyle l}}_j}\right),{{\displaystyle d}}^{\max }$	Deployed bandwidth of ${{\displaystyle l}}_j$, remaining bandwidth of physical link ${{\displaystyle e}}_i$, delay of the physical link ${{\displaystyle e}}_{{{\displaystyle l}}_j}$, and maximum delay.
	${{\displaystyle B\left({{\displaystyle v}}_y\right)}}^{\max },{{\displaystyle T}}_r,\delta$	Maximum bandwidth, resume time, and downtime threshold.
Decision variable	$\left[{{\displaystyle T}}_C^1,{{\displaystyle T}}_C^2\right]$	Estimated time section.
	${{\displaystyle T}}_d\left({{\displaystyle s}}_i\right),{{\displaystyle T}}_e\left({{\displaystyle s}}_i\right),{{\displaystyle T}}_a\left({{\displaystyle u}}_i\right)$	Downtime, the migration completion time of ${{\displaystyle s}}_i$, and actual arrival time of corresponding ${{\displaystyle u}}_i$.
	$Fn,F$	Migration failure rate and migration failure number.
	${{\displaystyle \epsilon }}_z^x,{{\displaystyle \epsilon }}_j^i$	Binary variables.
	${{\displaystyle A,H,L\left({{\displaystyle v}}_y\right)}}^{\min },{{\displaystyle L\left({{\displaystyle v}}_y\right)}}^{\max }$	Migration path set, candidate migration path set, and shortest and longest length in $A$.
	$MP\left({{\displaystyle v}}_y\right),L\left({{\displaystyle v}}_y\right)$	Migration path and its length.
	${{\displaystyle CB\left({{\displaystyle v}}_y\right)}}^{\min },{{\displaystyle CB\left({{\displaystyle v}}_y\right)}}^{\max }$	Minimum and maximum migration bandwidth meeting the constraints.
	$PL\left({{\displaystyle v}}_y\right),B\left({{\displaystyle v}}_y\right)$	Migration path load and migration bandwidth.
	${{\displaystyle T}}_m\left({{\displaystyle v}}_y\right),{{\displaystyle T}}_e\left({{\displaystyle v}}_y\right),{{\displaystyle {{\displaystyle T}}_e\left({{\displaystyle v}}_y\right)}}^{\prime }$	Decision migration time, decision completion time, and actual completion time.

summarizes the notations used in this paper.

3.1. Problem Statement

In this paper, we study the routing and bandwidth allocation of mobile-aware SFC migration in the user estimable movement scenario. In mobile user networks, we need to migrate SFC in time to follow the movement of users to maintain service continuity. Moreover, it is necessary to improve the migration performance by using the user movement characteristics and designing the RBA algorithm. Given a physical network topology, several mobile-aware SFCs (i.e., SFC parameters), several mobile users (i.e., users’ known path and actual arrival time), and the normal distribution of user arrival time and confidence level (i.e., migration completion time constraint), we decide (1) VNFs to migrate, (2) migration routing, and (3) migration bandwidth, intending to (1) reduce the user waiting time for the migration completion and (2) reduce the migration failure rate. Via the known information, we put forward the softer migration migrating the mobile-aware SFC before the user reaches the corresponding access node. Combined with several constraints, the RBA problem is solved through path load adaptation to make multiple SFC migrations completed in the estimated time section and realize a low failure rate of migration. Below, we conclude all the assumptions in the paper.

• Physical network: We assume that each node has the same attributes, and each link has the same bandwidth [10].
• Mobile user: According to the central limit theorem [29], we assume that the user arrival time to the destination node follows the normal distribution. We also assume that users entering the network follow a Poisson distribution [12]. In most cases, users only request one service; thus, we assume that one SFC provides services for one user.
• Mobile-aware SFC: Because the migration time is affected by the memory of VNFs, we assume that the memory of VNFs obeys a uniform distribution [13,14]. To simplify, we assume that each migrated VNF of the same SFC starts to migrate at the same time. Although sharing a VNF instance (VNFI) among SFCs can save resources and reduce the number of migrations, the limited location of VNFIs can lead to the SFC having a larger delay and occupying more links. The authors of [10,12] did not consider sharing VNFI; thus, we assume that VNFI cannot be shared by more than one SFC.

Several assumptions can simplify the problem proposed in the paper without considering the complex situation. We will further study the complex scenarios for these assumptions in our future work.

3.2. Physical Network Model

The physical network model consisting of nodes and links is represented as ${{\displaystyle G}}^P=\left({{\displaystyle N}}^P,{{\displaystyle E}}^P\right)$, as shown in Figure 1. The set of physical nodes is ${{\displaystyle N}}^p=\left\{{{\displaystyle n}}_1,{{\displaystyle n}}_2,\dots ,{{\displaystyle n}}_{\left|PN\right|}\right\}$, where $\left|PN\right|$ is the number of physical nodes, and the set of physical links is ${{\displaystyle E}}^p=\left\{{{\displaystyle e}}_1,{{\displaystyle e}}_2,\dots ,{{\displaystyle e}}_{\left|PE\right|}\right\}$, where |PE| is the total number of physical links.

Each ${{\displaystyle n}}_i\in {{\displaystyle N}}^P$ has specific node attributes, consisting of CPU $P\left({{\displaystyle n}}_i\right)$, memory $M\left({{\displaystyle n}}_i\right)$, and storage $S\left({{\displaystyle n}}_i\right)$. The remaining CPU, storage, and memory resources of ${{\displaystyle n}}_i$ are $RP\left({{\displaystyle n}}_i\right)$, $RS\left({{\displaystyle n}}_i\right)$, and $RM\left({{\displaystyle n}}_i\right)$. SFC deployment must meet the requirement that the allocated node resource cannot exceed the maximum available resource.

The attributes of ${{\displaystyle e}}_i\in {{\displaystyle E}}^P$ are link bandwidth $B\left({{\displaystyle e}}_i\right)$ and delay $d\left({{\displaystyle e}}_i\right)$. The remaining bandwidth of ${{\displaystyle e}}_i$ is $RB\left({{\displaystyle e}}_i\right)$. If the link cannot provide the deployment or migration bandwidth, the SFC cannot be deployed or migrated, resulting in a migration failure. Similarly, the deployment fails if the physical link does not meet the delay constraint when deploying the SFC.

3.3. User Estimable Movement Scenario

The set of mobile users in the network is $U=\left\{{{\displaystyle u}}_1,{{\displaystyle u}}_2,\dots ,{{\displaystyle u}}_{\left|MU\right|}\right\}$. In Figure 1, the user is connected to the physical network by the physical node ${{\displaystyle n}}_1$; thus, the user’s current access node is ${{\displaystyle n}}_1$, and the user’s movement path is $\left\{{{\displaystyle n}}_1,{{\displaystyle n}}_5,{{\displaystyle n}}_6\right\}$, corresponding to the set of access nodes of SFC. However, the time when the user arrives at the access node is uncertain.

We discuss the user estimable movement scenario, characterized by the following:

• The user’s movement path is known;
• The user’s arrival time is regular and estimable.

For mobile users, the arrival time is affected by many factors which are independent and not decisive. The normal distribution is an excellent description of some variables in human society. Moreover, user arrival time is a continuous random variable. According to the central limit theorem, user arrival time is approximately a normal distribution. As shown in Figure 2, the mean of the user arrival time to the destination node ${{\displaystyle T}}_u$ is $150T$, the standard deviation is 30, and the actual arrival time ${{\displaystyle T}}_a\sim N(150T,{{\displaystyle 30}}^2)$.

Moreover, pre-copy mode transfers dirty pages to the destination server via multiple rounds to ensure the latest data when using the destination server. Even if the migration meets the stop-and-copy condition before the user arrives, we still need to continue the migration and wait for the user arrival to make sure the data are up to date, which results in unnecessary resource occupation. When the user arrives, if the migration does not meet the downtime condition, the user needs to wait for the migration completion, causing loss to QoS. Therefore, it is essential to narrow the actual user arrival time and migration completion time. In the user estimable movement scenario, we establish the estimation model of user arrival time to obtain the estimated arrival time $\left[{{\displaystyle T}}_C^1,{{\displaystyle T}}_C^2\right]$. Applying confidence interval of normal distribution, we calculate ${{\displaystyle T}}_C^1$ and ${{\displaystyle T}}_C^2$ as follows:

$${{\displaystyle T}}_C^1={{\displaystyle T}}_u-Z(C)\cdot \sigma ,$$

(1)

$${{\displaystyle T}}_C^2={{\displaystyle T}}_u+Z(C)\cdot \sigma ,$$

(2)

where ${{\displaystyle T}}_u$ and $\sigma$ are the mean and standard deviation of normal distribution, $C$ is the confidence level (possibility of the actual arrival time within the confidence interval), and $Z(C)$ is the value determined by $C$. Set $C$ = 30, 50, 70, and 90, where a higher confidence level results in a greater estimated arrival time (i.e., confidence interval of arrival time) in Figure 2. Using the estimation model of user arrival time, we set the migration time within the estimated time section to reduce the user waiting time for the migration completion.

3.4. SFC Migration Request Model

The set of SFCs is $S=\left\{{{\displaystyle s}}_1,{{\displaystyle s}}_2,\dots ,{{\displaystyle s}}_{\left|SF\right|}\right\}$, where the number of SFCs is $\left|SF\right|$. $\left|SF\right|$ is equal to $\left|MU\right|$. We express ${{\displaystyle s}}_i\in S$ as ${{\displaystyle G}}^{i,S}=\left({{\displaystyle N}}^{i,S},{{\displaystyle E}}^{i,S},{{\displaystyle A}}^{i,S}\right)$, where ${{\displaystyle N}}^{i,S}=\left\{{{\displaystyle v}}_1,{{\displaystyle v}}_2,\dots ,{{\displaystyle v}}_{\left|SN\right|}\right\}$ represents the set of VNFs, ${{\displaystyle E}}^{i,S}=\left\{{{\displaystyle l}}_1,{{\displaystyle l}}_2,\dots ,{{\displaystyle l}}_{\left|SN\right|-1}\right\}$ represents the set of links, and ${{\displaystyle A}}^{i,S}=\left\{{{\displaystyle a}}_1,{{\displaystyle a}}_2,\dots ,{{\displaystyle a}}_{\left|SA\right|}\right\}$ depicts the set of access nodes corresponding to the movement path of the user. In Figure 1, the set of access nodes consists of ${{\displaystyle n}}_1$, ${{\displaystyle n}}_5$, and ${{\displaystyle n}}_6$, $\left|SA\right|=3$.

Moreover, the SFC migration request involves multiple VNF migrations belonging to the same SFC, and each VNF is migrated in a pre-copy way. We express the set of migrated VNFs for ${{\displaystyle s}}_i$ in one movement as ${{\displaystyle N}}_m^{i.S}=\left\{{{\displaystyle v}}_1,{{\displaystyle v}}_2,\dots ,{{\displaystyle v}}_{\left|SM\right|}\right\}$. According to the pre-copy characteristics, the VNF migration request is expressed as $\left({{\displaystyle M}}_s,{{\displaystyle M}}_d,D,G,Q,C\right)$, where ${{\displaystyle M}}_s$ is the source node, ${{\displaystyle M}}_d$ is the destination node, $D$ is the memory dirtying rate, $G$ is the memory size of the migrated VNF, $Q$ is the distribution of user arrival time, and $C$ is the confidence level. Given the migration request, we perform the RBA algorithm to decide migration routing and bandwidth; then, the migration begins.

3.5. Problem Formulation

We express the problem of routing and bandwidth allocation of mobile-aware SFC migration in the user estimable movement scenario with the mathematical model below. The mathematical model applies to each migration caused by the user movement.

(1)

Objective function: The first objective is to reduce the user waiting time for the migration completion. To simply, when the user’s access node is changed, the extended link is not considered to provide service. If the corresponding SFC is not migrated to the changed node, the service will be interrupted; hence, the user waiting time for migration completion is also the service downtime. Then, the second objective function is the failure rate of SFC migration. Reducing the migration failure rate can effectively maintain service stability and improve users’ QoS in the user estimable movement scenario.

$$\min {{\displaystyle T}}_d\left({{\displaystyle s}}_i\right)=\min ({{\displaystyle T}}_e\left({{\displaystyle s}}_i\right)-{{\displaystyle T}}_a\left({{\displaystyle u}}_i\right)),\forall {{\displaystyle s}}_i\in S,\exists {{\displaystyle u}}_i\in U,$$

(3)

$$\min F=\min (\frac{Fn}{\left|SF\right|}),$$

(4)

where ${{\displaystyle T}}_d\left({{\displaystyle s}}_i\right)$, ${{\displaystyle T}}_e\left({{\displaystyle s}}_i\right)$, and ${{\displaystyle T}}_a\left({{\displaystyle u}}_i\right)$ are the user waiting time, the migration completion time of s_i, and the actual time when the corresponding user u_i reaches the target node, respectively. F is the failure rate of SFC migrations, F_n is the migration failure number, and |SF| is the number of SFCs. In one migration, s_i migration involves multiple VNF migrations. ${{\displaystyle T}}_e\left({{\displaystyle s}}_i\right)$ is subject to the maximum completion time of VNF migration in Equation (5),

$${{\displaystyle T}}_e\left({{\displaystyle s}}_i\right)=\underset{{{\displaystyle v}}_y\in {{\displaystyle N}}_m^{i,S}}{\max }{{\displaystyle T}}_e\left({{\displaystyle v}}_y\right),$$

(5)

where ${{\displaystyle T}}_e\left({{\displaystyle v}}_y\right)$ is the completion time of VNF ${{\displaystyle v}}_y$ migration. Each ${{\displaystyle v}}_y\in {{\displaystyle N}}_m^{i,S}$ migration needs to satisfy the constraints below.

(2)

Constraints: Different deployment locations of the original chain and the prepared chain trigger the mobile-aware SFC migration. In Figure 1, the original SFC’s VNFs are deployed in n₁, n₂, n₃, and n₄, the prepared SFC’s VNFs are deployed in n₅, n₂, n₃, and n₄, and VNF1 is migrated from n₁ to n₅. Although this paper focuses on migration strategy, the original and prepared chains of the SFC still need to be deployed in the algorithm. Thus, we need to consider the constraints of deployment and migration.

Node resource constraints of deployment: When the VNFs of the original and prepared chains are deployed on the physical nodes, the CPU, storage, and memory for each VNF must be less than the remaining CPU, storage, and memory of the physical node.

$${{\displaystyle \epsilon }}_z^x\cdot P\left({{\displaystyle v}}_z\right)\le RP\left({{\displaystyle n}}_x\right),\forall {{\displaystyle v}}_z\in {{\displaystyle N}}^{i,S},{{\displaystyle n}}_x\in {{\displaystyle N}}^P,$$

(6)

$${{\displaystyle \epsilon }}_z^x\cdot S\left({{\displaystyle v}}_z\right)\le RS\left({{\displaystyle n}}_x\right),\forall {{\displaystyle v}}_z\in {{\displaystyle N}}^{i,S},{{\displaystyle n}}_x\in {{\displaystyle N}}^P,$$

(7)

$${{\displaystyle \epsilon }}_z^x\cdot M\left({{\displaystyle v}}_z\right)\le RM\left({{\displaystyle n}}_x\right),\forall {{\displaystyle v}}_z\in {{\displaystyle N}}^{i,S},{{\displaystyle n}}_x\in {{\displaystyle N}}^P,$$

(8)

where binary variable ${{\displaystyle \epsilon }}_z^x$ is set to 1 if VNF ${{\displaystyle v}}_z$ is deployed on the physical node ${{\displaystyle n}}_x$. The CPU, storage, and memory resources of ${{\displaystyle v}}_z$ are $P\left({{\displaystyle v}}_z\right)$, $S\left({{\displaystyle v}}_z\right)$, and $M\left({{\displaystyle v}}_z\right)$. The remaining CPU, storage, and memory resources of node ${{\displaystyle n}}_x$ are $RP\left({{\displaystyle n}}_x\right)$, $RS\left({{\displaystyle n}}_x\right)$, and $RM\left({{\displaystyle n}}_x\right)$, respectively.

Bandwidth and delay constraints of deployment: While the links of original and prepared chains are deployed to the physical links, the deployed bandwidth of each virtual link cannot exceed the remaining bandwidth of the corresponding physical link. Moreover, the SFC’s delay should be less than the specified maximum delay.

$${{\displaystyle \epsilon }}_j^i\cdot b\left({{\displaystyle l}}_j\right) \le RB\left({{\displaystyle e}}_i\right), \forall {{\displaystyle l}}_j\in {{\displaystyle E}}^{i,S},{{\displaystyle e}}_i\in {{\displaystyle E}}^P,$$

(9)

$${\displaystyle \sum _{j=1}^{\left|SN\right|-1}d\left({{\displaystyle e}}_{{{\displaystyle l}}_j}\right)}\le {{\displaystyle d}}^{\max },$$

(10)

where binary variable ${{\displaystyle \epsilon }}_j^i$ is set to 1 if virtual link ${{\displaystyle l}}_j$ is deployed on the physical link ${{\displaystyle e}}_i$$,$ $b\left({{\displaystyle l}}_j\right)$ is the deployed bandwidth of ${{\displaystyle l}}_j$, $RB\left({{\displaystyle e}}_i\right)$ is the remaining bandwidth of ${{\displaystyle e}}_i$, $d\left({{\displaystyle e}}_{{{\displaystyle l}}_j}\right)$ is the delay of physical link ${{\displaystyle e}}_i$ where ${{\displaystyle l}}_j$ is deployed, and ${{\displaystyle d}}^{\max }$ is the maximum delay allowed by the corresponding service.

Bandwidth constraint of migration: A migration bandwidth greater than the memory dirtying rate is the convergence condition of pre-copy, and bandwidth must be less than the maximum available bandwidth of migration path,

$$D<B\left({{\displaystyle v}}_y\right)\le {{\displaystyle B\left({{\displaystyle v}}_y\right)}}^{\max },\forall {{\displaystyle v}}_y\in {{\displaystyle N}}_m^{i,S},$$

(11)

$${{\displaystyle B\left({{\displaystyle v}}_y\right)}}^{\max }=\underset{{{\displaystyle e}}_z\in MP\left({{\displaystyle v}}_y\right)}{\min }RB\left({{\displaystyle e}}_z\right),\forall {{\displaystyle v}}_y\in {{\displaystyle N}}_m^{i,S},$$

(12)

where $B\left({{\displaystyle v}}_y\right)$ is denoted as the VNF ${{\displaystyle v}}_y$ migration bandwidth, $MP\left({{\displaystyle v}}_y\right)$ is the migration path, and the maximum bandwidth ${{\displaystyle B\left({{\displaystyle v}}_y\right)}}^{\max }$ is the minimum value among the remaining bandwidth of each path segment in $MP\left({{\displaystyle v}}_y\right)$.

Downtime constraint of pre-copy: The sum of the last iteration time of the pre-copy and the resume time is less than the set threshold,

$${{\displaystyle T}}_{ml}\left({{\displaystyle v}}_y\right)+{{\displaystyle T}}_r\le \delta ,\forall {{\displaystyle v}}_y\in {{\displaystyle N}}_m^{i,S},$$

(13)

where ${{\displaystyle T}}_{ml}\left({{\displaystyle v}}_y\right)$ is the last iteration time of the VNF v_y migration, and T_r is the time to stop the source server and start the target server after copy completion.

Completion time constraint of migration: We compute the migration time using Equation (14). The migration completion time must be within the estimated time section.

$${{\displaystyle T}}_m({{\displaystyle v}}_y)=\frac{G}{B({{\displaystyle v}}_y)}\cdot \frac{1-{{\displaystyle \left(\frac{D}{B({{\displaystyle v}}_y)}\right)}}^{n+1}}{1-\left(\frac{D}{B({{\displaystyle v}}_y)}\right)},\forall {{\displaystyle v}}_y\in {{\displaystyle N}}_m^{i,S},$$

(14)

$${{\displaystyle T}}_e({{\displaystyle v}}_y)={{\displaystyle T}}_s({{\displaystyle v}}_y)+{{\displaystyle T}}_m({{\displaystyle v}}_y),\forall {{\displaystyle v}}_y\in {{\displaystyle N}}_m^{i,S},$$

(15)

$${{\displaystyle T}}_C^1\le {{\displaystyle T}}_e({{\displaystyle v}}_y)\le {{\displaystyle T}}_C^2,\forall {{\displaystyle v}}_y\in {{\displaystyle N}}_m^{i,S},$$

(16)

where ${{\displaystyle T}}_m\left({{\displaystyle v}}_y\right)$ is the migration time of VNF ${{\displaystyle v}}_y$, $n$ is iteration number of pre-copy, $D$ is memory dirtying rate, and $G$ is the VNF memory value. ${{\displaystyle T}}_s\left({{\displaystyle v}}_y\right)$ is the start time of VNF ${{\displaystyle v}}_y$ migration. ${{\displaystyle T}}_C^1$ and ${{\displaystyle T}}_C^2$ are the left and right limits of the estimated arrival time.

4. Approach to the Problem

4.1. Softer Migration

If using soft migration, we start to migrate the SFC to continue to provide services for users after users arrive at the corresponding access node, leading to a long time that the user waits for the migration completion. The migration starts with two conditions for soft migration: the last migration has been completed, and the user has arrived at the corresponding access node. In Figure 3a, after the user arrives at server 5′s node, we start the first migration, i.e., of VNF1 from server 1 to server 5 and of VNF2 from server 2 to server 4. Figure 4a shows network load changes for the first migration after the user’s access node (server 5’s node). Furthermore, the intervals between adjacent migrations are sometimes enormous. That is because, even after we have completed the last migration, we perform the new migration after the user reaches the access node of the current movement, e.g., starting the second migration of VNF1 after the user reaches server 7.

To provide services with short downtime, the key aspect of the softer migration strategy is establishing a prepared chain via the known mobile path before the user reaches the corresponding access node. For softer migration, migration initiation needs to meet two conditions: the last migration has been completed, and the user will reach the corresponding access node. In Figure 3b, while the user is at server 1′s node and will move to server 5′s node, we establish the prepared chain in advance to reserve resources. There exist the original chain and prepared chain of this SFC in the network. Then, we start multiple VNF migrations between the original chain and the prepared chain. Figure 4b indicates that network load changes due to the first migration when the user enters the network and is about to move. Furthermore, the intervals between adjacent migrations are minimal because the migration does not end until the user reaches the destination node, e.g., completing the migration of the VNF1 from server 1 to server 5 after the user reaches server 5.

The similarity between the two migration strategies is that they both adopt the pre-copy mode. In Figure 5, the dirtied memory data are continuously copied and transferred from the source node to the destination node. The iteration time of each round tends to decrease to convergence. After copying data, we release the VNFs on the source node and start the VNFs on the destination node.

4.2. Path Load Adaptive Routing and Bandwidth Allocation (PLARBA) Algorithm

This paper takes a mobile-aware SFC as a unit to explore multiple VNF migrations belonging to the same SFC. The migration algorithm of the overall process can be divided into four parts, as shown in Figure 6 and described in detail in Algorithm 1.

Algorithm 1. Migration algorithm of overall process

Input: Physical network ${{\displaystyle G}}^{i,S}=({{\displaystyle N}}^{i,S},{{\displaystyle E}}^{i,S},{{\displaystyle A}}^{i,S})$, the set of SFCs $S$, the set of users $U$, each SFC ${{\displaystyle G}}^{i,S}=({{\displaystyle N}}^{i,S},{{\displaystyle E}}^{i,S},{{\displaystyle A}}^{i,S}),\forall i\in \{1,2,3,\dots ,\left|SF\right|\}$, each user’s movement path and arrival time distribution $Q$, confidence level $C$, memory dirtying rate $D$.

Output: Number of SFCs failed to migrate $Fn$, failed SFC set $FSFC$, network migration record $Mrecord$.

1 $Fn\leftarrow 0$, $FSFC\leftarrow \left\{\right\}$, $Mrecord\leftarrow \left\{\right\}$.

2 for each user ${{\displaystyle u}}_i\in U$ do

3   for each movement do

4    According to Q and C, obtain estimated time section $\left[{{\displaystyle T}}_C^1,{{\displaystyle T}}_C^2\right]$

5     The last migration completion triggers the migration decision of SFC ${{\displaystyle s}}_i$ in this movement.

6     Call Algorithm 2 to obtain ${{\displaystyle N}}_m^{i,S}$, source and destination nodes.

7     for each migrated VNF $v_y\in {{\displaystyle N}}_m^{i,S}$ do

8      Obtain migration request $({{\displaystyle M}}_s,{{\displaystyle M}}_d,D,G,Q,C)$.

9      Execute Algorithm 3 to obtain $MP\left({{\displaystyle v}}_y\right)$, $B\left({{\displaystyle v}}_y\right)$, ${{\displaystyle T}}_m\left({{\displaystyle v}}_y\right)$.

10     if $MP\left({{\displaystyle v}}_y\right)=\varnothing$ then

11       $Fn=Fn+1$.

12       Add ${{\displaystyle s}}_i$ to FSFC.

13       Delete the original chain and the prepared chain.

14       break.

15      else

16       Add $MP\left({{\displaystyle v}}_y\right)$, $B\left({{\displaystyle v}}_y\right)$, ${{\displaystyle T}}_m\left({{\displaystyle v}}_y\right)$ to $Mrecord$.

17      Perform migration immediately.

18      if migration meets downtime condition in Equation (13) then

19       if not receive ${{\displaystyle T}}_a\left({{\displaystyle u}}_i\right)$ then

20        Continue the migration to wait for the user.

21        Actual migration time ${{\displaystyle {{\displaystyle T}}_e({{\displaystyle v}}_y)}}^{\prime }={{\displaystyle T}}_a\left({{\displaystyle u}}_i\right)+{{\displaystyle T}}_r$.

22       else

23        Actual migration time ${{\displaystyle {{\displaystyle T}}_e({{\displaystyle v}}_y)}}^{\prime }={{\displaystyle T}}_e({{\displaystyle v}}_y)$.

24       Add ${{\displaystyle {{\displaystyle T}}_e({{\displaystyle v}}_y)}}^{\prime }$ to $Mrecord$.

25    Using Equations (3) and (5), calculate user waiting time ${{\displaystyle T}}_d\left({{\displaystyle s}}_i\right)$ in this movement, and add ${{\displaystyle T}}_d\left({{\displaystyle s}}_i\right)$ to $Mrecord$.

26    Release the original chain and start the prepared chain.

27  return $Fn$, $FSFC$, $Mrecord$.

(1)

Establish user estimable movement scenario to obtain the estimated time section of arrival time: After the user starts moving to the corresponding access node, on the basis of the normal distribution of user arrival time and confidence level, we obtain the estimated time section of actual arrival time $\left[{{\displaystyle T}}_C^1,{{\displaystyle T}}_C^2\right]$.

(2)

Obtain migrated VNFs, source nodes, and destination nodes: When the user starts to move, on the basis of the known access node, we deploy the prepared chain on the shortest path between the next access node and the output node. According to the original and prepared chain’s VNF deployment location, we obtain the migrated VNFs in this movement, the corresponding migration source nodes, and destination nodes in Algorithm 2. If the shortest link does not meet the deployment constraints in Equations (6)–(10), we regard it as a migration failure caused by deployment.

Algorithm 2. Deploying prepared chain

Input: corresponding SFC ${{\displaystyle s}}_i$ ${{\displaystyle G}}^{i,S}=({{\displaystyle N}}^{i,S},{{\displaystyle E}}^{i,S},{{\displaystyle A}}^{i,S})$$, Fn$, $FSFC.$

Output: $Fn$, $FSFC$, ${{\displaystyle N}}_m^{i,S}$, source and destination nodes.

1 According to ${{\displaystyle A}}_s$, obtain the next access node of ${{\displaystyle s}}_i$.

2 Deploy the prepared chain on the shortest path between the next access node and the output node.

3 if constraints in Equations (6)–(10) are not satisfied then

4   $Fn=Fn+1$.

5   Add ${{\displaystyle s}}_i$ to FSFC.

6   Delete the original chain.

7   break;

8 According to the original chain and prepared chain, determine which VNFs to migrate ${{\displaystyle N}}_m^{i,S}$, source nodes, and destination nodes.

9 return $Fn$, $FSFC$, ${{\displaystyle N}}_m^{i,S}$, source and destination nodes.

(3)

Implement PLARBA algorithm: We acquire the migration requests including the memory dirtying rate, memory size, and user information, and then input them into PLARBA algorithm. The PLARBA in Algorithm 3 mainly has two parts.

Algorithm 3. PLARBA

Input: Migration request $({{\displaystyle M}}_s,{{\displaystyle M}}_d,D,G,Q,C)$.

Output: Decision migration path $MP\left({{\displaystyle v}}_y\right)$, decision migration bandwidth $B\left({{\displaystyle v}}_y\right)$, and decision migration time ${{\displaystyle T}}_m\left({{\displaystyle v}}_y\right)$.

1 Candidate migration path set $H\leftarrow \left\{\right\}$.

2 Find path set $A$ between source node ${{\displaystyle M}}_s$ and destination node ${{\displaystyle M}}_d$.

3 for each path ${{\displaystyle A}}_i\in A$ do

4   If bandwidth ${{\displaystyle A}}_i$ meets constraints in Equations (11), (13), and (16) then

5    Add ${{\displaystyle A}}_i$ to $H$.

6 for each path ${{\displaystyle H}}_i\in H$ do

7   Record ${{\displaystyle H}}_i$’s length.

8 The path with the least length in H is the decision migration path $MP\left({{\displaystyle v}}_y\right)$.

9 Using Equations (17) and (18), obtain path load $PL\left({{\displaystyle v}}_y\right)$ and decision migration bandwidth $B\left({{\displaystyle v}}_y\right)$.

10 Using Equation (14), obtain decision migration time ${{\displaystyle T}}_m\left({{\displaystyle v}}_y\right)$.

11 return $MP\left({{\displaystyle v}}_y\right)$, $B\left({{\displaystyle v}}_y\right)$, ${{\displaystyle T}}_m\left({{\displaystyle v}}_y\right)$.

• Routing selection: We find all the paths between the source node and the destination node of migration according to the deployment of the original chain and the prepared chain. Then, we traverse the path set. Each path meeting the constraints of migration bandwidth, migration time, and downtime in Equations (11), (13), and (16) is added to the candidate path set. After that, the migration path with the least length in the candidate path set is the decision migration path. If no migration path meets the constraints, the migration fails.
• Bandwidth Allocation: When the path has more remaining bandwidth, more bandwidth should be allocated to make better use of resources and complete faster. Conversely, less migration bandwidth is allocated, which makes it possible to provide bandwidth for other migration tasks as they pass through the same paths, to avoid network congestion and migration failure. The authors of [15] only took the migration path length as the standard to measure the path load, where a shorter path length indicates that the allocated bandwidth is closer to the maximum bandwidth. In this paper, we propose the following indicators for measuring the path load: (i) the path length, and (ii) the bandwidth occupancy rate of each path segment. We use the normalization method to measure the path load using Equation (17),

  $$PL\left({{\displaystyle v}}_y\right)=\frac{L\left({{\displaystyle v}}_y\right)-{{\displaystyle L\left({{\displaystyle v}}_y\right)}}^{\min }}{{{\displaystyle L\left({{\displaystyle v}}_y\right)}}^{\max }-{{\displaystyle L\left({{\displaystyle v}}_y\right)}}^{\min }}+\partial \cdot \frac{{\displaystyle \sum _{{{\displaystyle e}}_z\in MP\left({{\displaystyle v}}_y\right)}R\left({{\displaystyle e}}_z\right)}}{L\left({{\displaystyle v}}_y\right)},\forall {{\displaystyle v}}_y\in {{\displaystyle N}}_m^{i,S},$$

  (17)

  where $L\left({{\displaystyle v}}_y\right)$ represents the migration path length of VNF ${{\displaystyle v}}_y$ migration task, ${{\displaystyle L\left({{\displaystyle v}}_y\right)}}^{\min }$ and ${{\displaystyle L\left({{\displaystyle v}}_y\right)}}^{\max }$ are the shortest and longest length in the path set, respectively. $R\left({{\displaystyle e}}_z\right)$ is the bandwidth occupation rate of each path segment in the migration path, i.e., the ratio of occupied bandwidth to total bandwidth, the minimum sum of $R\left({{\displaystyle e}}_z\right)$ is 0, and the maximum is $L\left({{\displaystyle v}}_y\right)$.

Affected by the path load, the migration bandwidth is inversely proportional to the load defined by the length and occupancy rate of the migration path. The higher the path load is, the less the migration bandwidth is allocated. Otherwise, the more the migration bandwidth is allocated. Upon setting the shortest path length to 2, the longest path length to 10, and the ∂ in Equation (17) to 0.5,

Table 2. Comparison of path load in this paper with that in [15].

Path Length	Bandwidth Occupation Rate	Load in [15]	Load in the Paper
2	[0.3, 0.3]	0	0.3
2	[0.6, 0.8]	0	0.7
4	[0.3, 0.3, 0.3, 0.3]	0.25	0.55
4	[0.5, 0.3, 0.1, 0.3]	0.25	0.55
5	[0.3, 0.3, 0.3, 0.3, 0.3]	0.375	0.675
5	[0.5, 0.3, 0.3, 0.2, 0.3]	0.375	0.695

shows that the calculated load in this paper is larger than that in [15]. If an available path is short but the remaining bandwidth is small, we obtain a high value using Equation (17) because of the high load. Occupied bandwidth is also an important indicator of migration path load. Therefore, we measure the path load more comprehensively to adjust migration bandwidth effectively. We calculate the decision migration bandwidth of each VNF by adjusting the path load as follows:

$$B\left({{\displaystyle v}}_y\right)={{\displaystyle CB\left({{\displaystyle v}}_y\right)}}^{\max }-\min \left(PL\left({{\displaystyle v}}_y\right),1\right)\cdot \left({{\displaystyle CB\left({{\displaystyle v}}_y\right)}}^{\max }-{{\displaystyle CB\left({{\displaystyle v}}_y\right)}}^{\min }\right),\forall {{\displaystyle v}}_y\in {{\displaystyle N}}_m^{i,S},$$

(18)

where $B\left({{\displaystyle v}}_y\right)$ represents the migration bandwidth of the VNF ${{\displaystyle v}}_y$ migration task. ${{\displaystyle CB\left({{\displaystyle v}}_y\right)}}^{\min }$ and ${{\displaystyle CB\left({{\displaystyle v}}_y\right)}}^{\max }$ are the minimum and maximum migration bandwidth meeting the constraints. Since migration bandwidth ranges from the maximum value to the minimum value, the adaptive coefficient needs to be $\le 1$.

(4)

Adopt path and bandwidth, perform the migration: We execute all VNF migration tasks for the SFC. If the migration has met the stop-and-copy constraint in Equation (13) before the user arrives, the migration continues to wait for the user, and the actual migration completion time T_e(v_y)′ is the sum of the user’s actual arrival time and resumption time of pre-copy; otherwise, it is the decision migration completion time in Equation (14). After the user reaches the destination node and all VNF migrations are completed, we release the original chain and start the prepared chain.

$${{\displaystyle {{\displaystyle T}}_e\left({{\displaystyle v}}_y\right)}}^{\prime }=\{\begin{array}{l}{{\displaystyle T}}_a\left({{\displaystyle u}}_i\right)+{{\displaystyle T}}_r \mathrm{if} \mathrm{migration} \mathrm{meet} \mathrm{constraint} \mathrm{in} \mathrm{Equation} \left(13\right) \hfill \\ {{\displaystyle T}}_e\left({{\displaystyle v}}_y\right)\hspace{1em}\hspace{1em}\hspace{1em}\mathrm{otherwise}\hfill \end{array},\forall {{\displaystyle v}}_y\in {{\displaystyle N}}_m^{i,S}.$$

(19)

Obviously, the complexity levels of PLARBA and Algorithm 1 are $O\left({{\displaystyle m}}_A+{{\displaystyle m}}_H\right)$ and $O\left(\left|MU\right|\cdot {{\displaystyle a}}_{\left|SA\right|}\cdot {{\displaystyle a}}_{\left|SM\right|}\cdot \left({{\displaystyle m}}_A+{{\displaystyle m}}_H\right)\right)$, respectively, where ${{\displaystyle m}}_A$ is the number of paths in $A$, ${{\displaystyle m}}_H$ is the number of paths in $H$, ${{\displaystyle a}}_{\left|SA\right|}$ is the average number of user movements (i.e., the average number of SFC access nodes), ${{\displaystyle a}}_{\left|SM\right|}$ is the average number of migrated VNFs per migration per SFC.

5. Simulation Results

5.1. Simulation Setting

We developed a migration simulation platform under the user estimable movement scenario on the basis of a general simulation platform of NFV resource allocation SFCSim [30]. We simulated the NSFNET network with 14 nodes and 21 links. The CPU, memory, and storage of each node were set to 300 GHz, 300 GB, and 300 GB, the bandwidth of each link was 4 GBps, and the link delay followed a normal distribution $N(1,{{\displaystyle 0.5}}^2)$ s.

Furthermore, we set 10 types of VNFs. The CPU and storage of each VNF were 1 GHz and 1 GB. Memory obeyed $U(1,3)$ GB. The period T in the network was 0.1 s. The number of users $\left|MU\right|$ increased from 20 to 90. The mean ${{\displaystyle T}}_u$ of user arrival time to the next node obeyed $U(14,16)$ s. Moreover, the number of access nodes of each SFC was uniformly distributed from 3 to 7. The access and output nodes were randomly selected from 14 nodes. For each SFC, the number of VNFs followed $U(2,6)$, the bandwidth of links followed $U(5,20)$ MBps, and the delay followed $U(0.4,0.8)$ s. Lastly, $\partial$ in Equation (17) was set to 0.5, memory dirtying rate was 50 MBps, maximum downtime was 1 s, and the resumption time of pre-copy ${{\displaystyle T}}_r$ was 0.2 s.

5.2. Softer Migration and Soft Migration with One SFC

Firstly, to verify the advantage of softer migration in reducing user waiting time, we implemented the softer migration strategy (SEM) compared with soft migration (SM) and considered the average user waiting time for migration completion as the performance indicator. Both migration strategies adopted the PLARBA algorithm. Then, we studied the influence of confidence level and standard deviation on this indicator. In the comparison experiment of softer migration and soft migration, we ran each algorithm 10 times, set a different SFC each time, and took the average value as the final result. Because of the low network load, the decision migration time was close to the left limit of the estimated time section.

(1)

The influence of confidence level: Figure 7a depicts that, compared with SM, the user waiting time of SEM was significantly reduced by 94.3% for $C=30$, 95.4% for $C=50$, 96.5% for $C=70$, and 97.7% for $C=90$. As confidence increased, the user waiting time of the two strategies gradually decreased. For the SEM algorithm, the waiting time for $C=90$ was 69.6% less than that for $C=30$. For the SM algorithm, the indicator for $C=90$ was 25.7% less than that for $C=30$. This is because, as accuracy increased, ${{\displaystyle T}}_C^1$ and the decision migration time decreased. However, the migration could not enter the stop-and-copy stage until the user arrived, and the actual migration time approached the user arrival time. Thus, the user waiting time approached ${{\displaystyle T}}_r$ gradually.

(2)

The influence of standard deviation: In Figure 7b, compared with SM, SEM reduced the user waiting time by 97.9% for $\sigma =1$, 98.0% for $\sigma =10$, 98.3% for $\sigma =20$, and 97.7% for $\sigma =30$. With standard deviation increasing, the indicators of the two strategies showed a falling trend. For the SEM algorithm, the waiting time for $\sigma =30$ was 22.6% less than that for $\sigma =1$. For the SM algorithm, the waiting time at $\sigma =30$ was 30.9% less than that for $\sigma =1$. Similarly, with standard deviation increasing, ${{\displaystyle T}}_C^1$ also reduced, leading to the user waiting time approaching ${{\displaystyle T}}_r$.

The simulation results fully prove the advantages of migration in advance in reducing user waiting time. SEM can reduce the waiting time by more than 90% at different confidence levels and standard deviations, reducing downtime and improving QoS. This section of the experiment with one SFC is the basis for PLARBA experiment with multiple SFCs. In the next section, to get a larger estimated time section, the confidence level is set to 90, and the standard deviation is 30.

5.3. PLARBA Algorithm and Two Comparison Algorithms with Multiple SFCs

To validate the advantages of the PLARBA algorithm on migration performance, we adopted the SEM strategy and the PLARBA algorithm compared with other algorithms. Furthermore, since PLARBA was an improvement of the DARBA algorithm in [15], we set DARBA and MAXRBA in [15] as the comparison algorithms. The difference among the three algorithms lies in how to allocate migration bandwidth.

DARBA: The migration bandwidth is only adjusted by the path length, $PL\left({{\displaystyle v}}_y\right)=\frac{L\left({{\displaystyle v}}_y\right)-{{\displaystyle L\left({{\displaystyle v}}_y\right)}}^{\min }}{{{\displaystyle L\left({{\displaystyle v}}_y\right)}}^{\max }-{{\displaystyle L\left({{\displaystyle v}}_y\right)}}^{\min }},\forall {{\displaystyle v}}_y\in {{\displaystyle N}}_m^{i,S}$, taking no account of the bandwidth occupation rate of each path segment.

MAXRBA: The maximum bandwidth value of the migration path is allocated satisfying the constraints as the migration bandwidth.

In the comparison experiment of PLARBA, DARBA, and MAXRBA, we explored the impact of the number of users on performance and considered five performance indicators:

• Average migration bandwidth: The average migration bandwidth allocated for all requests.
• Network load maximum: The maximum sum of occupied bandwidth in physical links from when the first user enters the network to when no users are in the network.
• The number of SFC migrations: The total migration times of the SFCs without migration failure and the existing migration times of the SFCs with midway migration failure.
• The number of VNF migrations: The total number of VNF migrations from when the first user enters the network to when there are no users.
• Failure rate of SFC migration: Ratio of the number of SFCs with migration failure to the number of SFCs.

(1)

Average migration bandwidth: As shown in Figure 8a, the average migration bandwidth of PLARBA was significantly lower than that of the two comparison algorithms because it considered the path length and the path bandwidth occupancy. In addition, when the number of SFCs changed, the differences in migration bandwidth were within the range [1.5, 2.6] MB/T. When there were 80 SFCs, the bandwidth of PLARBA decreased by 2.6 MB/T compared with that of DARBA. As more and more SFCs were introduced into the network, the migration bandwidth tended to decrease. Compared with the case of 20 SFCs, the indicator of 90 SFCs was reduced by 3.5% using PLARBA, 1.3% using DARBA, and 0.3% using MAXRBA. When the number of SFCs increased, the maximum bandwidth satisfying the constraints decreased and the adaptive coefficient of PLARBA was larger, leading to less allocated bandwidth.

(2)

Network load maximum: As shown in Figure 8b, when there were 20 SFCs in the network, the indicator of PLARBA was 362.8 MB/T less than that of DARBA and 426.9 MB/T less than that of MAXRBA. When more users were set in the network, the load value of PLARBA was higher than that of the two comparison algorithms because PLARBA could migrate more SFCs successfully through load adaption. When 90 SFCs were considered, the indicator of PLARBA increased by 156.8 MB/T compared to DARBA and 472.7 MB/T compared to MAXRBA. The network load values of the three algorithms increased with the number of users. For PLARBA, the load value with 90 SFCs was 135.1% higher than that with 20 SFCs compared to 103.6% for DARBA and 90.0% for MAXRBA. Regardless of the algorithm, the increasing trend gradually slowed down due to the limited capacity of the network.

(3)

The number of SFC migrations: Figure 9a displays that PLARBA had more SFC migrations compared with the two other algorithms. When 50 SFCs were set, the value of the PLARBA algorithm increased by 25.2% and 20.7% compared with DARBA and MAXRBA, respectively. When 70 SFCs were set, the value of the PLARBA algorithm increased by 24.8% and 24% compared with DARBA and MAXRBA, respectively. When the number of SFCs increased from 20 to 90, the differences in migration times among the three algorithms increased gradually. When the network had more than 50 SFCs, the differences remained stable. More users in the network led to more SFC migrations. Similarly, the number of SFC migrations increased to the saturation point.

(4)

The number of VNF migrations: As shown in Figure 9b, the number of VNF migrations when adopting the PLARBA algorithm was 2–3 times that of the corresponding SFC migrations because each SFC migration caused two or three VNF migrations. When considering 50 SFCs, the value of the PLARBA algorithm increased by 18.9% and 17.1% compared with DARBA and MAXRBA, respectively. When considering 70 SFCs, the value of the PLARBA algorithm increased by 16.5% and 22.7% compared with DARBA and MAXRBA, respectively. As the number of SFCs increased from 20 to 50, the difference in this indicator among the three algorithms gradually increased. When considering more than 50 SFCs, the difference mainly remained in a specific range. Similar to the number of SFC migrations, compared with 20 SFCs, the number of VNF migrations for the PLARBA algorithm with 90 SFCs increased by 140.8%, the indicator for DARBA increased by 100%, and the indicator for MAXRBA increased by 123.8%.

(5)

Failure rate of SFC migration: Figure 10 shows that the migration failure rate of PLARBA was always higher than that of the two comparison algorithms, indicating that considering the path length and bandwidth occupancy can significantly reduce the indicator. When considering 50 SFCs in the network, compared with DARBA, the failure rate of PLARBA decreased by 75.4%, and the decline was the most obvious. Comparing PLARBA and DARBA, the failure rate of PLARBA with 70 SFCs decreased by 42.9%, while the failure rate of PLARBA with 90 SFCs decreased by 22.4%. When the number of SFCs increased, the difference among the three algorithms first increased and then decreased. More SFCs were set when fewer users were in the network, more SFCs were migrated successfully, and the advantage of load adaptation was more pronounced. However, when the number of users was large, the remaining bandwidth of the physical link was less, and the load adaption ability of PLARBA was limited.

According to the experimental simulation under a different number of users, PLARBA allocated the bandwidth dynamically according to the path load, allowing more SFCs to migrate successfully, while better utilizing network bandwidth and obtaining a lower migration failure rate. Therefore, PLARBA can provide continuous service for more mobile users and PLARBA is especially more appropriate when there are not too many users.

6. Conclusions

In dynamic networks with mobile-aware SFCs, the user estimable movement scenario with a known movement path and estimable arrival time is common. To solve the migration problem in this scenario, utilizing the known movement information, we put forward a softer migration strategy to migrate the SFC before the user reaches the corresponding access node. Then, for routing and bandwidth allocation issues, to reduce the number of SFCs with migration failure, we proposed the PLARBA algorithm taking path load as the bandwidth allocation factor. Then, we conducted experiments to verify the superiority of the proposed algorithm in the NSFNET network. The comparison of SEM and SM showed that the softer migration strategy could significantly reduce the user waiting time by more than 90%. The comparison of PLARBA, DARBA, and MAXRBA revealed that PLARBA decreased the total migration failure rate by up to 75%, resulting in more continuous and stable services for more mobile users. The proposed algorithm in this paper can be applied to mobile networks involving vehicles to provide continuous service by using the regularity of movement.

In future work, we plan to formalize the user estimable movement model instead of using the qualitative normal distribution and apply a neural network to effectively predict users’ actual arrival time. Moreover, we will consider sharing VNFI among SFCs and decide when to start the migration. In addition, we will optimize the deployment algorithm to further improve the migration success rate. Furthermore, we will coordinate the migration bandwidth of all SFCs in the network to fully use bandwidth resources and further reduce the migration failure rate.