Cooperative Strategies in Two-Echelon Rescue Delivery Environment with Accessibility Uncertainty

Abstract

Cooperation between rescue teams is important to improve rescue performance. Vehicles outside of the disaster area usually deliver rescue resources. A two-echelon rescue delivery model is proposed, considering the isolated island of the disaster area where all the roads to the outside are interrupted. This paper first presents a non-cooperation scenario and then a cooperation scenario in an uncertain environment. Furthermore, two types of cooperative strategies to improve rescue performance are provided in the paper. The two cooperative strategies are a reactive cooperative strategy and an anticipatory cooperative strategy. Numerical experiments are used to evaluate the rescue performances of the two cooperation strategies by comparing them with the non-cooperation scenario. The results reveal that the anticipatory cooperative strategy performs the best in different cases varying in size.

1. Introduction

Cooperation in disaster response is critical considering that catastrophes such as earthquakes, floods, and hurricanes have devastating effects beyond human injuries and property damages. Furthermore, disaster response to catastrophes requires the involvement of multiple organizations, such as governments and Non-Governmental Organizations (NGOs).

However, cooperation among multiple organizations has to face the challenges of a variety of uncertainties. Especially, designing a cooperation strategy is still challenging under an uncertain environment, resulting in an incomplete view of the collaborative vehicle routing problem in uncertain environments. Assuming that stochastic demands are known only when the vehicle arrives at the customer location, Zhu (2014) [1] explored the impact of cooperation strategy on potential benefits, and developed the paired cooperative re-optimization strategy. They found that the strategy outperformed the paired locally coordinated scheme. In sum, due to the challenges brought about by the enormous complexity, assessing collaboration potentials under uncertainties has received little attention in the literature [2]. In order to address this research gap, we make the following contributions in this article. First, we propose a new cooperation two-echelon rescue model under specially accessibility uncertainty. The two-echelon logistics system is a general mode of a distribution system [3]. In this study, we propose an uncertain two-echelon rescue delivery problem model for humanitarian relief operations under the “isolated islands effect”. Rescue resources are first distributed from outside to the Local Distribution Centers (LDCs) in disaster-affected areas, and then the resources are distributed from the LDCs to the disaster victims. When the repair times of interrupted roads are uncertain, designing a collaboration strategy is important to minimize the expected maximal relief time from outside to the disaster victims. In this research, we propose two cooperative strategies and compared their performances with the non-cooperative strategy. Although Noyan (2016) [4] explored accessibility uncertainty in a two-echelon rescue system, they just considered the network accessibility’s uncertainty. They did not consider the uncertainty of rescue starting time on an “isolated island”. In fact, isolated islands are often formed because of geological structure change and mud-rock flow due to an earthquake, and in this way interruption between an external safety zone and the interior disaster area is induced. When multiple roads to the disaster area are interrupted and all of these roads are repaired at the same time, if the time for each road to resume traffic is uncertain, a new problem of cooperation under this uncertainty will arise.

Second, we develop two cooperation strategies in uncertain rescue environments: the reactive cooperative strategy and the anticipatory cooperative strategy. These strategies can enable a rescue team to dynamically construct routes when another rescue team’s start time is uncertain. Third, we conduct numerical studies that show the benefits in terms of the expected rescue times of our cooperative strategies.

2. Literature Review

Horizontal cooperation can obtain more benefits than non-collaboration in varied deterministic environments [5]. Cruijssen (2007) [6] found that general logistics service providers strongly believe in the potential benefits of horizontal cooperation to increase their profitability. Perez-Bernabeu (2015) [7] compared a full cooperative scenario and a non-cooperative scenario and found 8.9–29.3% savings for a cooperative scenario via computational experiments of many instances. Cooperation among logistic suppliers can also help to reduce delivery or routing costs in different scenarios. In terms of distance-based costs as well as emission costs, the average gap between a collaborative strategy and a non-collaborative one with the clustered policy is about 5%, while the average gap between a collaborative strategy and a non-collaborative one with the scattered policy is up to 92%.

Although extra benefits can be obtained through cooperation, determining the proportion of gains (costs) for each participant still impedes the widespread adoption of horizontal collaboration. If a partner perceives its allocated share of the coalition cost to be too large, it might leave the coalition [8]. Additionally, Dudek and Stadtler (2007) [9] argued that if the incentives are correctly distributed, a solution can be obtained that is optimal for the total coalition instead of a solution that is locally optimal for only one partner or one subset of partners. Thus, there has been increasing research studying the cooperative mechanisms for horizontal collaboration, for example, cooperative mechanisms in which reward flexibility increases the consolidation gains [10]. Cooperation methods based on cooperative game theory, such as the Shapley value method, maximin core method, and the Equal Cost Saving Method (ECSM), resulted in different solutions in an asymmetric example [11]. By comparing a coalition that adopts the Shapley value cost allocation rule and one who adopts the compensation for non-delivery rule, Defryn (2016) [8] demonstrated that a decision made at the operational (routing) level should affect the cost allocation result and vice versa. Fernández (2018) [12] also found that a collaborative solution that adopts a compensating mechanism would encourage collaboration and save coalition cost. Thus, it is necessary to explore a mechanism to provide an incentive for each partner that benefits every member in the coalition. Chinh (2017) [13] proposed a new collaboration strategy that helps logistics service providers to reduce the transportation costs when they collaborate with each other. They found a fully collaborative strategy and a collaborative strategy in which serving one’s own customer first could save 24% and 22% of costs, respectively. The cooperative mechanism based on the Vehicle Routing Problem (VRP) is summarized in

Table 1. Cooperative mechanisms (strategies) based on the Vehicle Routing Problem (VRP) summary.

Problem Type	Stochastic Variable	Non-Cooperative Scenarios	Cooperative Scenarios	Cooperative Mechanisms (Strategies)	Authors
PCVRP (Periodic Capacitated Vehicle Routing Problem)			The partner relaxes its constraints, such as delivery dates, order sizes, and order splitting rules, in a coalition.	Cooperative mechanisms based on how to share the gained benefits. Especially, persuading companies to relax their delivery terms by rewarding flexibility.	Vanovermeire (2014)
Multi-depot VRP		Each player minimizes the total transportation cost of all their vehicles.	The players cooperate using each other’s vehicles in order to minimize their transportation costs.	Cooperative mechanisms based on how to assign the coalition cost. Especially, allocation depends on the amount that each player adds to a coalition, and they receive a percentage of cost-saving.	Zibaei (2016)
Selective VRP			Serving the customers of all partners in one single logistic operation to minimize compensation for non-delivery.	Cooperative mechanisms based on how to assign the coalition cost. Especially, partners that consistently set high compensation for non-delivery cost values for their customers should be penalized by being assigned a relatively large share of the total coalition cost.	Defryn, Sörensen, and Cornelissens (2016)
Shared customer CVRP		Consumers demand from every carrier only delivered by the carrier.	The demand for each shared customer can be delivered by more than one company.	Cooperative mechanisms based on how to reduce coalition costs compared with the scenario without collaboration.	Fernández (2018)
Multi-VRP				Strategy 1: LSPs (logistics service providers) fully collaborate with each other. Strategy 2: an LSP’s vehicle can serve customers from other LSPs only after they finish serving their customers.	Chinh et al. (2017)
VRPSD (Vehicle Routing Problem with Split Deliveries)	demands		A single pair of vehicles cooperates to serve a set of customer demands.	PCR (paired cooperative reoptimization): multiple customers are dynamically assigned when each vehicle completes the current assignment, and trigger a sharing of information. PLC (paired locally coordinated): each pair of vehicles serve customers sequentially in opposite directions, following a fixed route, and if one vehicle fails, the remaining customers are assigned to its partner.	Zhu (2014)
MDVRPSD (MDVRP with Stochastic Demands)	Vehicle breakdowns; traveling, loading, and unloading times	Each manufacturer distributes its own products with its own car	Each manufacturer’s products can be distributed through vehicles of other manufacturers or third-party logistics companies		Sprenger (2012)
MDVRPSD (MDVRP with Stochastic Demands)	Stochastic demands	Each company plans the optimal routing plans.	Companies are involved in the cooperation agreement, which includes the sharing of storage areas and vehicle capacities.		Quintero-Araujo (2016)
Two-echelon VRP	Accessibility		The vehicles of team 0 and team 1 work together to deliver supplies to LDCs, and the vehicles of LDCs work together to deliver supplies to victims.		This paper

.

In uncertain environments, it is intuitive that cooperation can obtain more benefits than non-collaboration. Sprenger and Möonch (2012) [14] confirmed that a cooperative scenario outperforms a non-cooperative one in a stochastic environment. However, they did not explore different cooperation strategies. Similarly, Quintero-Araujo et al. (2016) [15] compared the solutions for a cooperative scenario with a non-cooperative scenario. They found that the expected total costs and the expected reliability of the cooperative scenario are better than the non-cooperative scenario. However, they did not explore cooperative strategies in detail and only employed the marginal saving as the main rule for potential customer allocation.

3. Problem Formulation

3.1. Basic Background

Accessibility uncertainty is one type of uncertainty in humanitarian relief. We consider a specific accessibility uncertainty, which often appears in earthquakes. For example, after the 2008 Wenchuan Earthquake in China, the town of Yingxiu was completely disconnected from the world and at least 12,000 people were trapped in it. In this situation, the “isolated islands effect” appeared because of non-accessibility to the disaster area. Consequently, a new cooperative problem was encountered by the rescue organizations to repair the interrupted roads.

In this paper, the term isolated island refers to the non-accessibility of the disaster area from the outer area. The non-accessibility occurs because the roads connecting to the disaster area are broken. The repaired road is uncertain because of many uncertainties, such as assessment of the damage and availability of resources. We assumed that the island is connected by two roads A and B, which need repair after the disaster. If interrupted road A is first repaired while interrupted road B is still in maintenance, rescue team A has to consider how to cooperate with rescue team B in the uncertain environment, as shown in Figure 1.

It is necessary to rapidly repair the broken road to transport relief materials to the people when the “isolated islands effect” appears. At the same time, the rescue teams at every interrupted road should adopt a reasonable cooperation strategy to deliver relief materials (e.g., food and water) in the shortest time to all the survivors in the isolated island.

In sum, the research question of this paper is based on a new Two-Echelon Vehicle Routing Problem (2E-VRP). A two-echelon vehicle routing problem is commonly found in logistics enterprises and express delivery service companies for their multimodal freight transportation and hypermarkets products distribution. Research has been done on developing heuristic methods to resolve 2E-VRP [16,17,18,19,20,21,22]. Furthermore, there have been further studies based on 2E-VRP, such as a time-dependent two-echelon capacitated vehicle routing problem, and a two-echelon vehicle routing problem with simultaneous pickup and delivery [23,24,25]. A detailed literature review can be found in [26].

In general, the two-echelon vehicle routing problem is a distribution system where intermediate depots, called satellites or Local Distribution Centers (LDCs), are placed between a supplier and final customers. In this paper, we assume that there are two suppliers (i.e., rescue team) in the system. Each supplier (i.e., rescue team) delivers rescue resources through one road to the LDC. At the same time, the two roads accessible to the isolated island are interrupted simultaneously. The time it takes to restore the two damaged roads to their original status is independently distributed on $\left[0,T{S}_i\right],i=A,B$. Once the damaged roads are restored to their original status, rescue teams distribute these goods to LDCs, and the LDCs distribute these goods to victims using their vehicles.

Before presenting our model formally, we first list our assumptions regarding travel from the rescue teams to the LDCs:

• There are $n_s$ LDCs and $n_c$ disaster victims in the isolated islands.
• Each LDC is only served by one rescue team’s vehicle.
• When the vehicle of any rescue team is on the way from one LDC to the next LDC, which is determined by a cooperative strategy, it is not allowed to take another new LDC as the next destination during transportation.

For the second echelon of travel from the LDCs to the disaster victims, our assumptions are as follows:

• The demand for each disaster victim is known and cannot be split between the vehicles of different LDCs.
• The demands of disaster victims cannot be delivered by direct shipping from the rescue team but must be consolidated in one of the LDC.
• Each disaster victim is served by one LDC vehicle.
• All LDC vehicles have the same capacity limitations.
• The amount delivered by each vehicle cannot exceed its capacity.
• Each LDC vehicle must begin and end at the same LDC.
• The rescue resources received by an LDC are completely delivered to the disaster victims.
• The overall capacity of all LDCs can fulfill the total demand.

We can then formulate the model as follows:

$$min\overline{\lambda }={\displaystyle \sum }_{\psi \in \left\{{\psi }_1,\cdots {\psi }_{2n_s}\right\}}{P}_{\psi }{\lambda }^{max}\left(\psi \right)$$

(1)

$${\lambda }_k^A\left(\psi \right)=X+{\delta }_k^A\left(\psi \right),\forall k\in V_{As}$$

(2)

$${\lambda }_k^B\left(\psi \right)=Y+{\delta }_k^B\left(\psi \right),\forall k\in V_{Bs}$$

(3)

$${\delta }_k\left(\psi \right)\le {\delta }^{max}\left(\psi \right),\forall k\in V_s$$

(4)

$${\delta }_k^A\left(\psi \right)=T_k^A\left(\psi \right)+{\eta }^k\left(\psi \right),\forall k\in V_{As}$$

(5)

$${\delta }_k^B\left(\psi \right)=T_k^B\left(\psi \right)+{\eta }^k\left(\psi \right),\forall k\in V_{Bs}$$

(6)

$$T_k^A\left(\psi \right)={\displaystyle \sum }_{k\in V_s}{t}_{Al}{x}_{Al}\left(\psi \right)+{\displaystyle \sum }_{k\in V_{As}}{\displaystyle \sum }_{l\in V_{As}}{t}_{lk}{x}_{lk}\left(\psi \right),\forall k,l\in V_{As}$$

(7)

$$T_k^B\left(\psi \right)={\displaystyle \sum }_{k\in V_s}{t}_{Bl}{x}_{Bl}\left(\psi \right)+{\displaystyle \sum }_{k\in V_{Bs}}{\displaystyle \sum }_{l\in V_{Bs}}{t}_{lk}{x}_{lk}\left(\psi \right),\forall k,l\in V_{Bs}$$

(8)

$${\eta }^k\left(\psi \right)={\displaystyle \sum }_{j\in V_{ck}}{t}_{kj}^k{y}_{kj}^k\left(\psi \right)+{\displaystyle \sum }_{i\in V_{ck}}{\displaystyle \sum }_{j\in V_{ck}}{t}_{ji}^k{y}_{ji}^k\left(\psi \right)+{\displaystyle \sum }_{i\in V_{ck}}{t}_{ik}^k{y}_{ik}^k\left(\psi \right),\forall k\in V_s$$

(9)

$${\displaystyle \sum }_{l\in V_s}{x}_{kl}\left(\psi \right)={\displaystyle \sum }_{l\in V_s}{x}_{lk}\left(\psi \right),\forall k\in V_s$$

(10)

$${\displaystyle \sum }_{j\in V_c}{y}_{ij}\left(\psi \right)={\displaystyle \sum }_{j\in V_c}{y}_{ji}\left(\psi \right),\forall k\in V_s,\forall i\in V_c$$

(11)

$${\displaystyle \sum }_{l\in V_s}{x}_{kl}\left(\psi \right)=1,\forall k\in V_s$$

(12)

$${\displaystyle \sum }_{j\in V_c}{y}_{ij}^k\left(\psi \right)=1,\forall i\in V_c$$

(13)

$${\displaystyle \sum }_{j\in V_c}{d}_j{y}_{ij}^k\left(\psi \right)=q,\forall k\in K,\forall i\in V_s\cup V_c$$

(14)

$$x_{kl}\left(\psi \right)\in \left\{0,1\right\},\forall k\in \left\{V_{s,}A,B\right\},\forall l\in V_s$$

(15)

$$y_{ij}^k\left(\psi \right)\in \left\{0,1\right\},\forall k\in V_{s,}\forall i\left\{V_s,V_c\right\},\forall j\in V_c$$

(16)

$$\psi \in \left\{{\psi }_1,\cdots {\psi }_{2n_s}\right\}$$

(17)

In the model above, we use the following notations:

Stochastic variables:

• $X$ is the time in which interrupted road A is restored to its original status.
• $Y$ is the time in which interrupted road B is restored to its original status.

Decision variables:

• $x_{kl}$ a binary variable of the first-level routing that equals 0 if LDC k is visited before LDC l.
• $y_{ij}^k$ a binary variable of the second-level routing that equals 1 if a second-level vehicle starts from LDC k and goes from disaster victim i to disaster victim j.
• $V_{ck}$ is a disaster victim set that is assigned to LDC k.
• $V_{is}$ is an LDC set that is assigned to the rescue team $\mathrm{i}\in \left\{A,\mathrm{B}\right\}$

Parameters:

• $V_{AB}=\left\{v_A,v_B\right\}$ Rescue team.
• $V_s=\left\{V_{s_1},V_{s_2},\cdots V_{s_{ns}}\right\}$ Set of local distribution centers; $n_s=$ number of LDCs.
• $V_c=\left\{V_c,V_{c_2},\cdots V_{c_{nc}}\right\}$ Set of disaster victims; $n_c=$ number of disaster victims.
• ${\lambda }_k^i$ Relief time from interrupted road $\mathrm{i}\in \left\{A,\mathrm{B}\right\}$ to victims through LDC k.
• ${\delta }_k^i$ Travel time from interrupted road $\mathrm{i}\in \left\{A,\mathrm{B}\right\}$ to victims through LDC k.
• $t_{Al}$ Travel time from interrupted road A to LDC l directly.
• $t_{Bl}$ Travel time from interrupted road B to LDC l directly.
• $T_k^A$ Travel time from interrupted road A to LDC k (through l).
• $T_k^B$ Travel time from interrupted road B to LDC k (through l).
• ${\eta }^k$ Travel time from LDC k to victims.
• $t_{ij}^k$ Travel time from disaster victim i to disaster victim j by vehicle k.
• $t_{kl}$ Travel time from LDC k to LDC l.
• $d_j$ Demands of disaster victim j.
• q Vehicle capacity.
• ${\lambda }^{max}$ Maximum rescue resource delivery time.
• ${\lambda }_k$ Rescue resource delivery time through LDC k.
• $\psi$ Stochastic restored time combination scenarios.
• $P_{\psi }$ Probability of time combination scenario $\psi$.
• $\overline{\lambda }$ Expected maximum rescue resource delivery time.
• ${\delta }_i$ Travel time from rescue team $\mathrm{i}\in \left\{A,\mathrm{B}\right\}$ to victims.

In the model, Constraints (2) and (3) demonstrate that the relief time includes the restored time and the travel time. Constraint (4) shows that the total travel time cannot exceed the maximal travel time in each stochastic restored time scenario. Constraint (5) shows that the travel time from rescue team A to LDC k $T_k^A$ and the routing time from LDC k to its corresponding disaster victims. Constraint (6) shows that the travel time from rescue team B to LDC k $T_k^B$ and the routing time from LDC k to its corresponding disaster victims. Constraint (7) shows that the travel time from A. Constraint (8) shows that the travel time from B. Constraint (9) shows that the routing time from LDC k to its corresponding disaster victim is set on the second level and back to LDC k.

The objective is to minimize the expected maximal relief time from the rescue teams to the disaster victims. As the total demand for all of the victims is known in our research, we normalize the quantity of each victim to be 1. Additionally, as every LDC tries its best to help local victims, it is appropriate to assume that any LDC vehicle is fully loaded with quantity q and distributes the rescue resources to q victims.

3.2. Research Question

In order to save as much time as possible, which victims need to receive resources first and the specific routing plan from LDCs to the victims must still be well planned. After discussing the routing at the second level, that is, from the LDCs to the victims, we return to the first level, from the rescue teams to the LDCs. At this point, another critical issue arises—how to ascertain vehicle routes when one interrupted road is restored and the other interrupted road is still in maintenance. This situation affects the minimal expected maximal relief time.

Simply put, if team A is responsible for all rescue tasks and rescue team B is ignored, this problem is a typical 2E-VRP. However, it is very difficult to make a decision, considering that it generally includes an infinite number of scenarios for stochastic scenario $\psi$. In fact, defining a clear cooperation strategy will simplify this problem. In the next section, we propose two cooperative strategies and compare their rescue performances. That is, the research question is focused on cooperative strategies, and cooperative strategies define one rescue team’s next delivery LDC and victim when another rescue team is in an uncertain environment.

To simplify the decision-making process, we only make new decisions in each LDC when the first level vehicles arrive at the LDC. Thus, infinite stochastic scenarios can be transformed into limited scenarios. We begin to make a decision when one depot restores traffic. Assume that road A is restored now, and the stochastic scenario of restoring road B includes the different time epochs encountered by vehicle using road A. The scenario includes the travel to the first LDC, travel from the first LDC to the second LDC, and so on, up to the last point. It generates a total of $n_s$ scenarios.

4. Non-Cooperation, Cooperation Strategies and Method

Strategic analysis exists in various fields, such as enterprise management, supply chain management, and rescue management [27,28]. In this section, we focus on non-cooperation and cooperation strategies analysis.

4.1. Non-Cooperation Situation

In a non-cooperative scenario, rescue teams (i.e., A and B) do not influence each other’s decisions. Rescue team A (rescue team B) will take the nearest rescue target-based rule to find the routing plan. That is, a rescue team will take the nearest LDC from the remaining LDCs as the next delivery target. At the same time, the vehicle of each LDC will take the nearest victim from remaining victims as the next delivery target once they get relief supplies when interrupted road A (interrupted road B) is restored first.

Rescue team A (rescue team B) will repeat the decision when it arrives at a new LDC, whether interrupted road B (interrupted road A) is restored or not. The arrangement is similar to the non-cooperation situation in MDVRP, where each player (owner) minimizes the total transportation costs of their own vehicle.

The non-cooperative situation can be considered as the worst-case scenario for relief operation. In uncertain situations, the worst case can be used as a benchmark to compare other policies, such as cooperative scenarios. It is a conservative strategy to perform some tasks that can be done in the shortest time in the current uncertain environment.

4.2. Cooperation Strategies

In a cooperative scenario, rescue team A and rescue team B must coordinate their efforts by considering the influence of the other team’s decision on the future rescue time. That is, rescue team A (rescue team B) must decide which LDC should be visited based on the evaluation of interrupted road B’s (interrupted road A’s) restored time when interrupted road A (interrupted road B) is restored first. Rescue team A (rescue team B) will repeat the decision when it arrives at a new LDC until interrupted road B (interrupted road A) is restored. When both roads are restored, the uncertain situation becomes a deterministic situation and both rescue teams deliver the remaining LDCs together according to the optimization solution.

We develop two cooperative strategies. One is the reactive cooperative strategy, which makes decisions without incorporating knowledge of the future. The other is the anticipatory cooperative strategy, which makes decisions incorporating information about the future.

Assuming that both rescue teams’ restored times follow certain distributions between $\left[0,T{S}_i\right],i=A,B$ independently, we focus on the situation in which interrupted road A is restored first, as the analysis is similar to the case where the interrupted road B is restored first. We explain the two strategies as follows:

(1) Reactive Cooperative Strategy (RCS)

The reactive cooperative strategy is based on the nearest delivery rule in an uncertain situation. Before road B is restored, rescue team A delivers its LDCs according to the nearest delivery rule. When the LDCs obtain their resources form rescue team A, the LDCs’ vehicles visit their victims according to the nearest delivery rule as well. When road B is restored, the problem becomes a deterministic 2E-VRP. In that situation, we can obtain the new optimized plan and ascertain the routing of remaining LDCs and victims according to the optimized method (see Appendix A).

(2) Anticipatory Cooperative Strategy (ACS)

The anticipatory cooperative strategy focuses on how to incorporate future information in the current decision. For the first echelon, when interrupted road A is restored at T₀ (the corresponding state is s₀), we know that interrupted road B will be restored between T₀ and TS_B. We will adopt the minimum expected value criterion and decide the first LDC of rescue team A before interrupted road B is restored by incorporating future information. When rescue team A reaches the first LDC, if interrupted road B has not been repaired, we repeat this decision process and determine the second LDC until interrupted road B is repaired. If interrupted road B is restored when rescue team A delivers resources from the origin to its first LDC, the problem becomes a deterministic 2E-VRP. We can re-optimize the problem for the remaining LDCs and obtain the new solution according to the algorithm described in Appendix A.

For the second echelon routing plan, we also adopt a minimum expected value criterion to choose the solution. That is, when we choose an LDC as the first LDC at s₀, we calculate all future states according to Appendix A and get the first and second echelon routing plan of all possible states. We choose the most likely state’s solution as the second echelon routing plan. When road B is restored, the problem becomes a deterministic 2E-VRP. In this situation, we can obtain the new optimized plan and ascertain the routing of remaining victims according to the algorithm described in Appendix A.

4.3. Method for Anticipatory Cooperative Strategy

We focus on the method to decide LDCs and victims according to ACS as RCS. The key to this method is how to incorporate future information in the current decision. In fact, incorporating future information is important for most of the dynamic and stochastic vehicle routing problems [29]. In recent years, approximate dynamic programming (ADP) has become a powerful framework for calculating the future impact of a decision and solving stochastic and dynamic problems typically arising in the field of operations research. In studies related to a vehicle routing problem, Novoa and Storer (2009) [30] examined an approximate dynamic programming algorithm for the single-vehicle routing problem with stochastic demands from a dynamic perspective and confirmed the efficient Rollout Algorithms (RAs) for solving the single VRPSD (Vehicle Routing Problem with Split Deliveries) under a dynamic approach. Ulmer and Voß (2016) [31] analyzed how risk-aversion impacted solutions’ quality and variance by integrating risk-aversion into ADP methods. Ulmer (2018) [32] incorporated temporal and spatial anticipation of service requests into Approximate Dynamic Programming (ADP) procedures to yield dynamic routing policies for the un-capacitated single-vehicle routing problem with stochastic service requests. Thus, in this work, we adopt a method based on an approximate dynamic programming framework to resolve cooperation problems based on anticipatory cooperative strategy.

A fundamental challenge in approximate dynamic programming is identifying an optimal action to be taken from a given state. In this work, we focus on LDC selection as an action via rollout algorithms, which are forward dynamic programming-based look-ahead procedures that estimate rewards-to-go through suboptimal policies [33].

In particular, we propose the method as follows. First, whether road A or B is repaired first, the decision process is the same. Without loss of generality, we assume that road A is repaired first. For the first echelon routing plan, we define the time point in which restored road A is repaired first as decision epoch 0, the time point that the rescue team A arrives at the first LDC as decision epoch 1, and so on. The last decision epoch K means that the interrupted road B will be restored when rescue team A delivers resources form the K^th LDC to the (K+1)^th LDC. The k^th decision epoch marks the beginning of the k^th period, at which time the system occupies state s_k in state space S, and at which time the decision-maker chooses LDC from the set of feasible A(s_k).

S_k means interrupted road B may be restored when rescue team A delivers resource to the (K+1)^th LDC or from the (K+1)^th LDC to the (K+2)^th, and so on, when rescue team A arrives at the K^th LDC. A state transition from state s_k in decision epoch k to state s_k+1 in decision epoch k + 1 is a function of the selected LDC and the set of random variables W_k+1 representing the random information whether interrupted road B may be restored or not between decision epochs k and k + 1. We denote the state transition as s_k+1 = S (s_k, a, W_k+1). As discussed in Powell (2011), we split the state transition into two parts—a transition from pre-decision state s_k to post-decision state s_k^a and a transition from s_k^a to pre-decision state s_k+1. We denote the deterministic transition to the post-decision state by the function S^A (s_k, a), which means that the rescue team A has chosen the LDC as the K^th LDC. The decision tree in Figure 2 provides a visual representation of the model elements. Square nodes represent pre-decision states, solid arcs depict actions, round nodes are post-decision states, and dashed arcs denote random information.

Because W_k+1 may be (not) realized when selecting LDC a and delivery rescue resource from LDC a to the (K + 1)^th LDC, we define the reward in decision epoch k as the expected reward, which denotes the expected rescue time for selected LDC a, and considering that interrupted road B may be restored in the future, and can be calculated by anticipatory optimization. For example, at decision epoch 0, if we adopt a⁰ as the first LDC of rescue team A, interrupted road B may be restored at rescue team A’s journey from the starting point to the first LDC, or from the first LDC to the second LDC, and so on. We calculate the final value of each possible state and obtain the expected value when a⁰ is the first LDC. We can also obtain the expected value when b⁰ is the first LDC, and so on. Finally, the LDC corresponding to the minimum expected value is taken as the first LDC.

When rescue team A arrives at the first LDC, if interrupted road B is not restored, we adopt the same decision process to identify the second LDC. If interrupted road B is restored when rescue team A arrived at the first LDC, we resolve the problem at decision epoch 1 by using the deterministic 2E-VRP algorithm.

5. Test Set and Results

5.1. Test Set

In this section, we use numerical experiments to illustrate the implications of cooperative strategies in a five-LDC disaster relief system. Without loss of generality, the instance set comprising 75 disaster victims was obtained from Gulczynski’s dissertation (

Table A1. The coordinates of disaster victims.

No.	x	y	No.	x	y	No.	x	y
1	22	22	26	41	46	51	29	39
2	36	26	27	55	34	52	54	38
3	21	45	28	35	16	53	55	57
4	45	35	29	52	26	54	67	41
5	55	20	30	43	26	55	10	70
6	33	34	31	31	76	56	6	25
7	50	50	32	22	53	57	65	27
8	55	45	33	26	29	58	40	60
9	26	59	34	50	40	59	70	64
10	40	66	35	55	50	60	64	4
11	55	65	36	54	10	61	36	6
12	35	51	37	60	15	62	30	20
13	62	35	38	47	66	63	20	30
14	62	57	39	30	60	64	15	5
15	62	24	40	30	50	65	50	70
16	21	36	41	12	17	66	57	72
17	33	44	42	15	14	67	45	42
18	9	56	43	16	19	68	38	33
19	62	48	44	21	48	69	50	4
20	66	14	45	50	30	70	66	8
21	44	13	46	51	42	71	59	5
22	26	13	47	50	15	72	35	60
23	11	28	48	48	21	73	27	24
24	7	43	49	12	38	74	40	20
25	17	64	50	15	56	75	40	37

). The coordinates of the 75 disaster victims are shown in Appendix C.

We also add the following information:

• The demand quantity of each disaster victim is 1.
• The capacity of vehicles at the second level is 15.

We get nine instances by setting the coordinates of the rescue teams, the LDCs’ coordinates, and the victims’ coordinates. For instance 1, we set the coordinates of the rescue team to (0, 0) and (50, 70), the coordinates of the LDCs to (20, 23), (60, 10), (18, 45), (45, 50), and (75, 40), and the coordinates of the victims are from Gulczynski’s dissertation. For instance 2, the coordinates of the rescue team and the victims are the same as in instance 1, except for the LDCs’ coordinates generated randomly within the range (x: 0–50, y: 0–70); for instance 3, we set the coordinates of the rescue team to (−20, −20) and (70, 90), the coordinates of the LDCs to (20, 23), (60, 10), (18, 45), (45, 50), and (75, 40), and the coordinates of the victims are generated randomly within the range (x: 0–50, y: 0–60); for instance 4, we set the coordinates of the rescue team to (−10, −10) and (60, 80), the coordinates of the LDCs are the same as in instance 3, and the coordinates of the victims are from Gulczynski’s dissertation. For instance 5, the coordinates of the rescue team and the victims are the same as in instance 4, except for the LDCs’ coordinates generated randomly within the range (x: 0–50, y: 0–70); For instance 6, the coordinates of the rescue team and the LDCs are the same as in instance 4, except for the victims’ coordinates, which are generated randomly within the range (x: 0–45, y: 0–45); for instance 7, the coordinates of the LDCs and the victims are the same as in instance 4, except for the rescue team coordinates, which are are set as (−30, 30) and (90, 30); for instance 8, the coordinates of the rescue team and the victims are the same as in instance 7, except for the LDCs coordinates, which are are set as (30, 40), (40, 20), (25, 50), (40, 40), and (78, 30); for instance 9, the coordinates of the rescue team are the same as in instance 8, the coordinates of the LDCs are the same as in instance 7, and the victim’s coordinates are generated randomly within the range (x: 0–50, y: 0–60).

5.2. Results

Assume that the vehicle travels at a speed of 60 kilometers per hour and that the distance unit between every two points is in kilometers. We present the non-cooperation and cooperation results when both interrupted roads’ restored times follow a uniform distribution between 0 and 2000 minutes (

Table 2. Results of the non-cooperation and cooperation strategies.

Instances	Non-Cooperation (NC)	Reactive Cooperation Strategy (RCS)	Anticipatory Cooperation Strategy (ACS)	RCS/NC	RCS Saving (1-RCS/NC)	ACS/NC	ACS Saving (1-RCS/NC)	ACS/RCS
Instance 1	611.82	298.09	277.21	0.49	0.51	0.45	0.55	0.93
Instance 2	892.78	279.02	258.06	0.31	0.69	0.29	0.71	0.92
Instance 3	1010.82	354.81	312.93	0.35	0.65	0.31	0.69	0.88
Instance 4	657.58	335.07	325.93	0.51	0.49	0.50	0.50	0.97
Instance 5	822.61	338.57	327.73	0.41	0.59	0.40	0.60	0.97
Instance 6	1135.63	361.51	342.78	0.32	0.68	0.30	0.70	0.95
Instance 7	465.00	293.12	272.57	0.63	0.37	0.59	0.41	0.93
Instance 8	505.19	283.66	256.06	0.56	0.44	0.51	0.49	0.90
Instance 9	483.74	274.28	263.23	0.57	0.43	0.54	0.46	0.96

).

From Table 2, we can find that the reactive cooperative strategy and anticipatory cooperative strategy are always better than the non-cooperation strategy. In terms of the expected maximal relief time, the reactive cooperative strategy (RCS) dominates its counterparts and can save 37–69% of the expected relief time compared to the non-cooperation strategy, and can save an average of 43.4% of the expected relief time compared to the non-cooperation strategy. The anticipatory cooperative strategy (ACS) dominates its counterparts and can save 41–71% of the expected relief time compared to the non-cooperation strategy (NC), and can save an average of 45.6% of the expected relief time compared to the non-cooperation strategy. At the same time, the anticipatory cooperative strategy (ACS) shows its advantage over an over-reactive cooperative strategy (RCS) from the last column, which suggests that designing better cooperation strategies can complete rescue missions more quickly in uncertain environments.

To illustrate the effect of cooperation strategies facing different uncertainties, we changed the range uniform distribution from [0, 100] to [0, 500]. The results are shown in

Table 3. Results from distribution interval change.

Instance	Description	Non-Cooperation (NC)	Reactive Cooperation Strategy (RCS)	Anticipatory Cooperation Strategy (ACS)	RCS/NC	RCS Saving (1-RCS/NC)	ACS/NC	ACS Saving (1-RCS/NC)
1	[50, 50]	561.97	250.36	236.72	0.45	0.55	0.42	0.58
2	[100, 100]	570.12	279.55	275.56	0.49	0.51	0.48	0.52
3	[150, 150]	636.04	298.09	286.99	0.47	0.53	0.45	0.55
4	[250, 250]	660.18	299.42	292.55	0.45	0.55	0.44	0.56
5	[300, 300]	682.33	328.67	319.98	0.48	0.52	0.47	0.53
6	[350, 350]	681.05	313.58	310.33	0.46	0.54	0.46	0.54
7	[400, 400]	735.98	353.52	320.40	0.43	0.57	0.42	0.58
8	[450, 450]	729.70	356.87	338.48	0.49	0.51	0.46	0.54
9	[500, 500]	739.77	372.38	344.70	0.50	0.5	0.47	0.53
Average						0.53		0.55

.

According to random repair times of road A and road B, we simulated the uncertain accessibility situation by using the process shown in Appendix B. From Table 3, we find that the reactive cooperative strategy (RCS) can save an average of 53% of the expected relief time compared to the non-cooperation strategy (NC). The anticipatory cooperative strategy (ACS) can save an average of 55% of the expected relief time compared to the non-cooperation strategy (NC), which suggests that the anticipatory cooperative strategy (ACS) is more effective than the reactive cooperative strategy (RCS), whether the uncertainty is high or low. The anticipatory cooperative strategy performs better because it can deal with uncertainty better before the operation of the other team begins.

Our numerical analysis also provides valuable managerial insights. For example, if decision-makers can smoothly incorporate the anticipatory cooperative strategy, it would dramatically improve disaster relief performance. In fact, management could benefit from the anticipatory cooperative strategy’s robustness in simplifying its decision process and execution procedures.

6. Conclusions

According to [15], a priori stochastic optimization means that random variables are modeled according to a probability function during the planning process, which is later not changed during the execution.

We also performed a priori stochastic control in each decision epoch based on the newly obtained information.

However, as new information changes are obtained in each decision epoch, we changed the remaining plans based on the already executed plans in the next decision epoch. In particular, we developed two cooperative strategies—the reactive cooperative strategy and the anticipatory cooperative strategy—to compare the performances.

According to the tests carried out using instances, the reactive cooperative strategy and anticipatory cooperative strategy can contribute to a reduction in total times for all of the considered instances. When considering a uniform distribution [0, 2000], the Reactive Cooperative Strategy (RCS) can save an average of 43.4% of the expected relief time compared to the non-cooperation strategy. The Anticipatory Cooperative Strategy (ACS) can save an average of 45.6% of the expected relief time compared to the non-cooperation strategy. When the range of uniform distribution changes from [0, 100] to [0, 500], the reactive cooperative strategy (RCS) can save an average of 53% of the expected relief time compared to the Non-Cooperation strategy (NC), and the Anticipatory Cooperative Strategy (ACS) can save an average of 55% of the expected relief time compared to the Non-Cooperation strategy (NC). Based on our extensive numerical experiments, we find that the anticipatory cooperative strategy is the most effective.

Our research is the first attempt to consider the cooperative strategies under the two-echelon vehicle routing problem with uncertain times needed for the restoration of interrupted roads. Hence, we focused on the feasibility and effectiveness of cooperative strategies. There are several possible extensions of this work that can be considered in future research. First, in our analysis, we only recommend two cooperation strategies. It would be meaningful to introduce other cooperative strategies in logistics and supply chains. Second, more stochastic issues can be considered, such as travel times between LDCs and victims. Third, cooperation vehicle routing problems in Autonomous Driving Vehicles (ADV) environments can be studied, considering that ADVs will develop further in the future [34,35].