Decomposition Methods for the Network Optimization Problem of Simultaneous Routing and Bandwidth Allocation Based on Lagrangian Relaxation

Abstract

The main purpose of the work was examining various methods of decomposition of a network optimization problem of simultaneous routing and bandwidth allocation based on Lagrangian relaxation. The problem studied is an NP-hard mixed-integer nonlinear optimization problem. Multiple formulations of the optimization problem are proposed for the problem decomposition. The decomposition methods used several problem formulations and different choices of the dualized constraints. A simple gradient coordination algorithm, cutting-plane coordination algorithm, and their more sophisticated variants were used to solve dual problems. The performance of the proposed decomposition methods was compared to the commercial solver CPLEX and a heuristic algorithm.

1. Introduction

As the latest reports say [1,2], during the first wave of the COVID-19 pandemic in the spring months of 2020, the global Internet traffic increased by almost 40%. This growth was prompted by telecommuting, sharp demand for video conferencing, online gaming, video streaming and social networking. It means that at the beginning of the pandemic crisis there was quite a big capacity surplus in almost all networks. In the same 2020 year, data transmission networks consumed 260–340 TWh or 1.1–1.4% of global electricity use [1]. It is expected that a strong growth in demand for the data centre and network services will continue at least until the end of this decade [3], particularly because of video streaming and gaming. For example, in 2022, they are supposed to make up 87% of consumer Internet traffic [1]. On the other hand, at the beginning of 2022, a global energy crisis has started and we have been observing unprecedented increase in electricity and energy commodities prices and the threat of their shortages. All this makes it more important than ever to use the network infrastructure efficiently to guarantee the highest possible quality of service with the lowest possible energy consumption.

The optimization problem taking into account these two criteria: a penalty for not achieving the eligible flow rates for connections and the total links cost was first formulated by Gallager and Golestaani as early as in 1980 [4]. The deep analysis of this model can be found in [5]. The authors assumed there a multi-path routing and transformed the problem to a standard multicommodity flow problem, which in the convex case could be solved efficiently by any continuous, nonlinear solver. Unfortunately, this approach is not applicable to modern data networks, since such networks have, as a rule, high fixed energy costs (even at low utilisation) and the link power cost cannot be well described by means of a continuous function [1]. Such more realistic formulations with binary variables have been presented in the last years in a connection with energy-aware aka green networks idea [6,7,8,9,10,11,12,13].

The first works on two-criteria network control, optimizing simultaneously both routing and flow rates, appeared in the 1990s [14,15,16]. Because routing variables are binary, these problems are of mixed-integer type, which means that they are NP-hard and as such for larger networks they require specialized suboptimal, heuristic algorithms to obtain an approximate solution in time acceptable for online application. Such methods were presented in the multi-path case in the papers [17,18,19] and in the single-path case in [14,15,16,20,21], all for the linear objective function case. Quite often, these heuristics used Lagrangian relaxation of the problem.

The precise model for quadratic problems, to be solved by a mixed-integer quadratic programming (MIQP) solver, such as CPLEX or Gurobi, was presented in the paper [22]. Unfortunately, the time of solving this problem and obtaining the exact solution is unacceptable in real-time systems. In this paper, the authors are trying to find an algorithm that delivers an approximate solution in short time.

This paper studies and analyzes different methods allowing for decomposing mixed-integer constrained nonlinear optimization problems using Lagrangian relaxation. Decomposition methods solve large-scale problems by splitting them into several smaller subproblems [23,24,25]. In most cases, they exploit the special structure of an optimization problem. A simple decomposable problem structure contains several independent subproblems, which are coupled with constraints and, as a result, the problem can be solved with a lower computational cost, since subproblems are usually much easier to solve than the original problem; moreover, the independence of these subproblems allows for a parallel solution.

Lagrangian relaxation was selected as a general decomposition technique allowing for transforming a problem having an additively separable form [23,24,26]. Usually, in discrete or mixed-integer nonlinear programming (MINLP), Lagrangian relaxation is used to retrieve a tight estimate of the optimal objective, and further this estimate can be used in a combination with other methods such as branch-and-bound. As a result of decomposition methods’ application, one obtains several independent subproblems (so-called local problems) and the master problem. They are solved iteratively, and the master problem provides parameters for local problems. These parameters are Lagrange multipliers. A deep study on the effectiveness of different optimization algorithms at the master level is presented in [21]. In the last decade, Lagrangian relaxation became a very popular approach to find a near-optimal solution in many practical MINLP problems, including: production planning [27,28], ecology [29], the electrical power industry [30], water distribution [31], supply chain networks [32,33,34], transportation [35,36], communication networks [37,38,39], and service systems [40].

In our work like in [21], both different versions of relaxations (dualizations) are analyzed and different approaches to solve the master (coordination) problem are used. We generalize the model from [16,21] because we analyze problems with a nonlinear cost function corresponding to QoS. One of our demand relaxations is quite similar to that of [16,21], but we do not dualize mixed constraints concerning individual arcs, what reduces the dimension of the coordination (dual) problem.

The paper is organized in the following way: Section 2 presents in detail a problem of simultaneous routing and bandwidth allocation in green networks, Section 3 discusses a general decomposition framework used in the paper, Section 4 contains detailed information about several approaches to the decomposition of the problem formulations presented in Section 2. In Section 5, coordination algorithms are introduced, and the results of the numerical tests for the decomposition methods are presented in Section 6. Section 7 summarizes and discusses the results.

2. Problem Formulation

The problem of the simultaneous routing and flow rate optimization can be described as finding single path routing and flow rates’ values satisfying network constraints for every source–destination pair [22].

The following items are used in the mathematical formulation of the problem:

$N,i\in N$	-	set of all network nodes and a single node, respectively;
$A,(i,j)\in A$	-	set of all network arcs and a single arc, respectively;
$G(N,A)$	-	network (graph) for which the optimization problem is formulated;
$L,l\in L$	-	set of all labeled links and a single labeled link, respectively;
$e:A\to L$	-	one-to-one mapping from arcs to links labeled by a single natural number;
$W,w\in W$	-	set of all demands and a single demand, respectively;
$s\left(w\right),d\left(w\right)$	-	source and destination node for the specific demand w, respectively;
$x_w$	-	flow rate for the specific demand w, $x_w\in R_{+}\cup \left\{0\right\}$,
${\underline{x}}_w,{\overline{x}}_w$	-	lower and upper bound on the flow rate for the demand w, we assume that ${\underline{x}}_w>0$;
$c_l$	-	capacity of the link l;
$b_{wl}$	-	binary routing decision variable, whether the link l is used by the demand w;
$\gamma$	-	positive parameter—weight of the QoS part of the objective function;
$\delta$	-	positive parameter—weight of the energy usage part of the objective function,

The problem can be formulated in the following way [22]:

$$P:\underset{x,b}{min}\sum _{w\in W}\left[\gamma {\left({\overline{x}}_w-x_w\right)}^2+\delta \sum _{l\in L}{b}_{wl}\right]$$

(1)

subject to

$$\forall w\in W,\forall j\in N,\sum _{\{i\in N|(i,j)\in A\}}{b}_{w,e(i,j)}-\sum _{\{k\in N|(j,k)\in A\}}{b}_{w,e(j,k)}=\left\{\begin{array}{cc}-1\hfill & j=s\left(w\right)\hfill \\ 1\hfill & j=d\left(w\right)\hfill \\ 0\hfill & otherwise\hfill \end{array}\right.$$

(2)

$$\begin{array}{cccc}\hfill & {\underline{x}}_w\le x_w\le {\overline{x}}_w,\hfill & \hfill & \forall w\in W\hfill \end{array}$$

(3)

$$\begin{array}{cccc}\hfill & \hspace{1em}\hspace{1em}\hspace{1em} b_{wl}\in \{0,1\}, \hfill & \hfill & \forall w\in W,\forall l\in L\hfill \end{array}$$

(4)

$$\begin{array}{cccc}\hfill & \sum _{w\in W}{b}_{wl}{x}_w\le c_l,\hfill & \hfill & \forall l\in L\hfill \end{array}$$

(5)

The above formulation will be later referred to as P. Objective function (1) is quadratic and convex. The first term is a Quality of Service component and expresses a cost for not delivering the full bandwidth for all connections $w\in W$. The second term expresses the total cost of energy used in the network and is proportional to the number of used links. The detailed description of this problem can be found in [22]. Other nonlinear, and even nonconvex, components of P are constraints (5). Because of the simultaneous usage of the binary variables $b_{wl}$ and continuous variables $x_w$, this optimization problem has a mixed domain. It means that P belongs to the MINLP class of the optimization problems and as such is NP-hard.

2.1. Linearized Constraints

The products of continuous and binary variables in nonlinear constraints (5), which are nonconvex functions, can be replaced with auxiliary variables $y_{wl}$ and several additional linear constraints [22]. The new, added linear constraints ensure fulfillment of the following equality:

$$y_{wl}=b_{wl}{x}_w,\forall w\in W,\forall l\in L$$

(6)

The optimization problem formulation with linearized constraints is presented below; this problem will be later referred to as $P_{cl}$:

$$\begin{array}{cc}\hfill P_{cl}:& \underset{x,b,y}{min}\sum _{w\in W}\left[\gamma {\left({\overline{x}}_w-x_w\right)}^2+\delta \sum _{l\in L}{b}_{wl}\right]\hfill \end{array}$$

(7)

subject to

$$\begin{array}{cc}\hfill \forall w\in W,\forall j\in N,& \sum _{\{i\in N|(i,j)\in A\}}{b}_{w,e(i,j)}-\sum _{\{k\in N|(j,k)\in A\}}{b}_{w,e(j,k)}=\left\{\begin{array}{cc}-1,\hfill & j=s\left(w\right)\hfill \\ 1,\hfill & j=d\left(w\right)\hfill \\ 0,\hfill & otherwise\hfill \end{array}\right.\hfill \end{array}$$

(8)

$$\begin{array}{cccc}\hfill & {\underline{x}}_w\le x_w\le {\overline{x}}_w,\hfill & \hfill & \forall w\in W\hfill \end{array}$$

(9)

$$\begin{array}{cccc}\hfill & b_{wl}\in \{0,1\},\hfill & \hfill & \forall w\in W,\forall l\in L\hfill \end{array}$$

(10)

$$\begin{array}{cccc}\hfill & \sum _{w\in W}{y}_{wl}\le c_l,\hfill & \hfill & \forall l\in L\hfill \end{array}$$

(11)

$$\begin{array}{cccc}\hfill & y_{wl}\ge x_w-{\overline{x}}_w(1-b_{wl}),\hfill & \hfill & \forall w\in W,\forall l\in L\hfill \end{array}$$

(12)

$$\begin{array}{cccc}\hfill & y_{wl}\ge 0,\hfill & \hfill & \forall w\in W,\forall l\in L\hfill \end{array}$$

(13)

$$\begin{array}{cccc}\hfill & y_{wl}\le x_w,\hfill & \hfill & \forall w\in W,\forall l\in L\hfill \end{array}$$

(14)

$$\begin{array}{cccc}\hfill & y_{wl}\le {\overline{x}}_w{b}_{wl},\hfill & \hfill & \forall w\in W,\forall l\in L\hfill \end{array}$$

(15)

Originally, constraints (12) and (15) used an arbitrarily selected so-called Big M number, that is a constant of a much higher order than the data of the problem [22], but, according to [41], replacing it with a smaller value may allow for achieving better numerical results. Thus, M was replaced with ${\overline{x}}_w$—the upper bound on the flow rate $x_w$.

2.2. Alternative Problem Formulation

An alternative problem formulation inspired by [42] may be proposed. This formulation still utilizes variables $y_{wl}$, but in this case flow conservation law Equations (2) are expressed with auxiliary variables $y_{wl}$. This problem formulation will be later referred to as $P_{alt}$:

$$\begin{array}{cc}\hfill P_{alt}:& \underset{x,b,y}{min}\sum _{w\in W}\left[\gamma {\left({\overline{x}}_w-x_w\right)}^2+\delta \sum _{l\in L}{b}_{wl}\right]\hfill \end{array}$$

(16)

subject to

$$\begin{array}{cc}\hfill \forall w\in W,\forall j\in N,\sum _{\{i\in N|(i,j)\in A\}}{y}_{w,e(i,j)}-\sum _{\{k\in N|(j,k)\in A\}}{y}_{w,e(j,k)}=& \left\{\begin{array}{cc}-x_w,\hfill & j=s\left(w\right)\hfill \\ x_w,\hfill & j=d\left(w\right)\hfill \\ 0,\hfill & otherwise\hfill \end{array}\right.\hfill \end{array}$$

(17)

$$\begin{array}{ccc}\hfill {\underline{x}}_w\le x_w\le {\overline{x}}_w,& \hfill & \forall w\in W\hfill \end{array}$$

(18)

$$\begin{array}{ccc}\hfill b_{wl}\in \{0,1\},& \hfill & \forall w\in W,\forall l\in L\hfill \end{array}$$

(19)

$$\begin{array}{cccc}\hfill & \sum _{\{j\in N|(i,j)\in A\}}{b}_{w,e(i,j)}\le 1,\hfill & \hfill & \forall w\in W,i\in N\hfill \end{array}$$

(20)

$$\begin{array}{cccc}\hfill & \sum _{w\in W}{y}_{wl}\le c_l,\hfill & \hfill & \forall l\in L\hfill \end{array}$$

(21)

$$\begin{array}{cccc}\hfill & y_{wl}\ge 0,\hfill & \hfill & \forall w\in W,\forall l\in L\hfill \end{array}$$

(22)

$$\begin{array}{cccc}\hfill & y_{wl}\le b_{wl}{\overline{x}}_w,\hfill & \hfill & \forall w\in W,\forall l\in L\hfill \end{array}$$

(23)

Constraints (20) force single path routing; constraints (23) keep the relationship between auxiliary variables and binary variables. Formulation $P_{alt}$ is equivalent to $P_{cl}$ and P provided that ${\underline{x}}_w>0,\forall w\in W$.

3. Lagrangian Relaxation

Lagrangian relaxation is one of the most popular relaxation methods [23,24]. In short, it consists in the replacement of the original optimization problem:

$$\underset{x\in X}{min}f\left(x\right)$$

(24)

subject to:

$$g\left(x\right)\le 0,$$

(25)

$$h\left(x\right)=0,$$

(26)

where $f:{\mathbb{R}}^n\mapsto \mathbb{R}$, $g:{\mathbb{R}}^n\mapsto {\mathbb{R}}^m,h:{\mathbb{R}}^n\mapsto {\mathbb{R}}^r,X\subseteq {\mathbb{R}}^n$ with a two-step problem

$$\underset{\lambda \ge 0,\mu =0}{max}\left\{L_D(\lambda ,\mu )=\underset{x\in X}{min}\left[L(x,\lambda ,\mu )=f\left(x\right)+{\lambda }^{T}g\left(x\right)+{\mu }^{T}h\left(x\right)\right]\right\}$$

(27)

The function $L(x,\lambda ,\mu )$ is called the Lagrange function or Lagrangian, the function $L_D(\lambda ,\mu )$ is called the dual function, the vectors $\lambda \in {\mathbb{R}}_{+}^m\cup \left\{0\right\}$, $\mu \in {\mathbb{R}}^r$ are called Lagrange multipliers. The external optimization in (27) is usually solved iteratively.

The theory says [43] that, if the set X is a convex subset of ${\mathbb{R}}^n$, the functions $f,g$ are convex over X, the functions $h\left(x\right)$ are affine, the optimal value $f^{*}$ is finite and there exists a vector $\overline{x}$ such that $g\left(\overline{x}\right)<0,h\left(\overline{x}\right)=0$, then there exist Lagrange multipliers such that the solutions of the primal (24)–(26) and the dual problem (27) are equal.

Lagrangian relaxation is particularly useful when the functions f and g are separable, that is, they can be expressed as the sums of the components dependent on the same separate subvectors of x, and the set X is a Cartesian product of the sets from spaces of the same subvectors. Such a structure of the optimization problem allows to approximate a large optimization problem by a set of smaller problems related to the original problem, solved in a loop together with a coordination problem delivering parameters modifying them.

Another example of a relaxation for the MINLP problem is continuous relaxation, which removes the integrality constraint and allows all variables to be real. The solution retrieved from the relaxation can be utilized in the branch-and-bound algorithm to obtain a solution of the original problem.

Lagrangian relaxation relies on constraints relaxation. For some problems having decomposable structure, it may result in significant problem simplification. The value of the dual function is always less than or equal to the optimal value of the primal problem. This property is called weak duality. Some problems may possess yet one property, called strong duality. For the strong duality, the optimal value of the dual problem is equal to the optimal value of the primal problem. The conditions for that are mentioned above. In such a case, Lagrangian relaxation allows for achieving the optimal value of the primal problem via solving the dual problem.

Unfortunately, an MINLP problem analyzed in this work operates on a mixed domain and only weak duality holds. It means that Lagrangian relaxation is able to provide only the lower bound to the primal problem.

Bounds provided by the Lagrangian relaxation are often used in conjunction with the branch-and-bound algorithm, and used in the same way as in the case of the continuous relaxation. However, usually Lagrangian relaxation provides tighter bounds than continuous relaxation [23]. At this point, it is important to introduce relations between primal optimal solution and dual optimal solution and their optimal decision variables.

Duality gap is the difference in the optimum between primal and dual problems’ objectives. It was noticed that in many practical problems the duality gap is vanishing when the number of variables grows. Evaluation of the duality gap is described in detail and proven in [44]. First of all, the duality gap depends on the number of relaxed constraints. This dependence can be described by the inequality:

$$inf\left(P\right)-sup\left(D\right)\le (m+1)E$$

(28)

where $inf\left(P\right)$ represents the optimal value of the primal problem objective function, $sup\left(D\right)$ represents the optimal value of the dual problem objective function and $inf\left(P\right)-sup\left(D\right)$ is the duality gap. The number of relaxed constraints is denoted by m and E is a parameter that depends on the problem objective function and its constraints specific properties. The estimation presented above is valid only if a list of assumptions is fulfilled [44]. Fortunately, the problem studied in this work satisfies these assumptions. We attempt to utilize the solution of the dual problem using special structure of the studied problem P. It allows for easy retrieving the feasible solution for situations when capacity constraints (5) were violated, but paths defined by routing variables still remain feasible. The method of retrieving such a feasible solution is described in Section 6.1 as a part of the heuristic algorithm.

4. Decomposition Methods

4.1. Demands Decomposition

The most obvious approach to apply Lagrange relaxation is based on the relaxation of the capacity constraints binding flows. There is no sense to consider the formulation P because of nonconvexity of the capacity constraints (5). Fortunately, for both $P_{cl}$ and $P_{alt}$ formulations, capacity constraints are represented by the same convex functions, respectively (11) and (21).

The Lagrange function for both $P_{cl}$ and $P_{alt}$ problems is as follows:

$$\begin{array}{c}\hfill L^{dem}(x,y,b,\lambda )=\sum _{w\in W}\left[\gamma {\left({\overline{x}}_w-x_w\right)}^2+\delta \sum _{l\in L}{b}_{wl}\right]+\sum _{l\in L}\left[{\lambda }_l\left(\sum _{w\in W}{y}_{wl}-c_l\right)\right]\end{array}$$

(29)

It is separable and, by changing the order of summation operators, we can write it as:

$$\begin{array}{c}\hfill L^{dem}(x,y,b,\lambda )=\sum _{w\in W}\left[\gamma {\left({\overline{x}}_w-x_w\right)}^2+\delta \sum _{l\in L}{b}_{wl}+\sum _{l\in L}{\lambda }_l{y}_{wl}\right]-\sum _{l\in L}{\lambda }_l{c}_l\end{array}$$

(30)

The dual function for the $P_{cl}$ problem is:

$$L_{Dcl}^{dem}\left(\lambda \right)=\underset{x,by}{min}{L}^{dem}(x,y,b,\lambda )$$

$$=\left\{\underset{[x_w,(y_{wl},b_{wl},l\in L),w\in W]}{min}\sum _{w\in W}\left[\gamma {\left({\overline{x}}_w-x_w\right)}^2+\delta \sum _{l\in L}{b}_{wl}+\sum _{l\in L}{\lambda }_l{y}_{wl}\right]\right\}-\sum _{l\in L}{\lambda }_l{c}_l$$

(31)

subject to (8)–(10) and (12)–(15).

Since, in the optimization problem (31), constraints (8)–(10) and (12)–(15) can be partitioned with respect to different flows $w\in W$ into completely independent, the minimization there can be performed independently for components dependent on particular flows w with respect to corresponding decision variables. That is, the dual function for $P_{cl}$ problem can be calculated in the following way:

$$\begin{array}{c}\hfill L_{Dcl}^{dem}\left(\lambda \right)=\sum _{w\in W}{L}_{cl,w}^{dem}\left(\lambda \right)-\sum _{l\in L}{\lambda }_l{c}_l\end{array}$$

(32)

where

$$L_{cl,w}^{dem}\left(\lambda \right)=\underset{x_w,[y_{wl},b_{wl},l\in L]}{min}\left[\gamma {\left({\overline{x}}_w-x_w\right)}^2+\delta \sum _{l\in L}{b}_{wl}+\sum _{l\in L}{\lambda }_l{y}_{wl}\right]$$

(33)

subject to

$$\begin{array}{cc}\hfill \forall j\in N& \sum _{\{i\in N|(i,j)\in A\}}{b}_{w,e(i,j)}-\sum _{\{k\in N|(j,k)\in A\}}{b}_{w,e(j,k)}=\left\{\begin{array}{cc}-1,\hfill & j=s\left(w\right)\hfill \\ 1,\hfill & j=d\left(w\right)\hfill \\ 0,\hfill & otherwise\hfill \end{array}\right.\hfill \end{array}$$

(34)

$$\begin{array}{cc}\hfill & {\underline{x}}_w\le x_w\le {\overline{x}}_w,\hfill \end{array}$$

(35)

$$\begin{array}{cccc}\hfill & b_{wl}\in \{0,1\},\hfill & \hfill & \forall l\in L\hfill \end{array}$$

(36)

$$\begin{array}{cccc}\hfill & y_{wl}\ge x_w-{\overline{x}}_w(1-b_{wl}),\hfill & \hfill & \forall l\in L\hfill \end{array}$$

(37)

$$\begin{array}{cccc}\hfill & y_{wl}\ge 0,\hfill & \hfill & \forall l\in L\hfill \end{array}$$

(38)

$$\begin{array}{cccc}\hfill & y_{wl}\le x_w,\hfill & \hfill & \forall l\in L\hfill \end{array}$$

(39)

$$\begin{array}{cccc}\hfill & y_{wl}\le {\overline{x}}_w{b}_{wl},\hfill & \hfill & \forall l\in L\hfill \end{array}$$

(40)

The dual function for $P_{alt}$ problem is defined as the optimization problem with the same objective function as for the problem $P_{cl}$, but with different constraints, that is:

$$L_{Dalt}^{dem}\left(\lambda \right)=\underset{x,by}{min}{L}^{dem}(x,y,b,\lambda )$$

$$=\left\{\underset{[x_w,(y_{wl},b_{wl},l\in L),w\in W]}{min}\sum _{w\in W}\left[\gamma {\left({\overline{x}}_w-x_w\right)}^2+\delta \sum _{l\in L}{b}_{wl}+\sum _{l\in L}{\lambda }_l{y}_{wl}\right]\right\}-\sum _{l\in L}{\lambda }_l{c}_l$$

(41)

subject to (17)–(20) and (22)–(23).

In the same way as for the problem $P_{cl}$, it may be calculated as follows:

$$L_{Dalt}^{dem}\left(\lambda \right)=\sum _{w\in W}{L}_{alt,w}^{dem}\left(\lambda \right)-\sum _{l\in L}{\lambda }_l{c}_l$$

(42)

where the component functions $L_{alt,w}^{dem}$ are

$$L_{alt,w}^{dem}\left(\lambda \right)=\underset{x_w,[y_{wl},b_{wl},l\in L]}{min}\left[\gamma {\left({\overline{x}}_w-x_w\right)}^2+\delta \sum _{l\in L}{b}_{wl}+\sum _{l\in L}{\lambda }_l{y}_{wl}\right]$$

(43)

subject to:

$$\forall j\in N,\sum _{\{i\in N|(i,j)\in A\}}{y}_{w,e(i,j)}-\sum _{\{k\in N|(j,k)\in A\}}{y}_{w,e(j,k)}=\left\{\begin{array}{cc}-x_w,\hfill & j=s\left(w\right)\hfill \\ x_w,\hfill & j=d\left(w\right)\hfill \\ 0,\hfill & otherwise\hfill \end{array}\right.$$

(44)

$$\begin{array}{cccc}\hfill & {\underline{x}}_w\le x_w\le {\overline{x}}_w,\hfill & \hfill & \hfill \end{array}$$

(45)

$$\begin{array}{cccc}\hfill & b_{wl}\in \{0,1\},\hfill & \hfill & \forall l\in L\hfill \end{array}$$

(46)

$$\begin{array}{cccc}\hfill & \sum _{\{j\in N|(i,j)\in A\}}{b}_{w,e(i,j)}\le 1,\hfill & \hfill & \forall i\in N\hfill \end{array}$$

(47)

$$\begin{array}{cccc}\hfill & y_{wl}\ge 0,\hfill & \hfill & \forall l\in L\hfill \end{array}$$

(48)

$$\begin{array}{cccc}\hfill & y_{wl}\le b_{wl}{\overline{x}}_w,\hfill & \hfill & \forall l\in L\hfill \end{array}$$

(49)

The dual problem for $P_{cl},P_{alt}$ formulations can be presented, respectively, as:

$$\begin{array}{c}\hfill D_{cl}=\underset{\lambda \ge 0}{max}{L}_{Dcl}^{dem}\left(\lambda \right)\end{array}$$

(50)

and

$$\begin{array}{c}\hfill D_{alt}=\underset{\lambda \ge 0}{max}{L}_{Dalt}^{dem}\left(\lambda \right)\end{array}$$

(51)

As it can be seen from the formulations presented above, in both cases, the problem was decomposed and the minimization, which is performed to determine the dual function, can be done independently for every demand $w\in W$.

Such a structure was achieved by the relaxation of the coupling capacity constraints. The number of relaxed constraints is equal to $\left|L\right|$. The number of variables presented in the studied problem, when no artificial variables $y_{wl}$ were used, is equal to $\left|W\right|\left(\right|L|+1)$. When $y_{wl}$ were used, the number of variables is equal to $\left|W\right|\left(2\right|L|+1)$. Hence, the relation of the number of relaxed constraints to the number of variables is acceptable.

Because capacity constraints were relaxed, optimal decision variables of the local problems may violate them. However, fortunately, the method of optimal feasible flow rate allocation, described in Section 6.1, can be applied for routes retrieved from the problem solution, to find a feasible solution.

4.2. Subnetworks Decomposition

Subnetworks decomposition is based on distinguishing subgraphs in the problem graph $G(N,A)$. These subgraphs, also called subnetworks, are selected groups of nodes, loosely connected with other nodes. The less the subnetwork is connected with other subnetworks, the tighter relaxation will be (that is, with the smaller number of the relaxed constraints). The new problem formulations presented in this subsection introduce subnetworks $g^m,m=1,\dots ,S$. Every node $n\in N$ belongs exactly to one subnetwork; arcs can belong to one or two subnetworks. Arcs which belong to a pair of subnetworks are duplicated for every subnetwork, and this encompasses their labeled links and flow rates. All duplicated components must be equal for keeping this formulation valid.

The following additional components are used in a mathematical problem description for the subnetworks decomposition, which is an adaptation of the original problems $P_{cl}$ or $P_{alt}$:

$\{g^1,g^2,\dots ,g^S\}$	-	division of a given network $G(N,A)$
		into S subnetworks that is $G(N,A)={\bigcup }_{m=1}^S{g}^m$; this is not a partition because of common links on the borders (see below),
$N^m$	-	set of all nodes of a subnetwork $g^m$ - this is a partition of the set N that is: $N={\bigcup }_{m=1}^S{N}^m$, $N_i\cap N_j=\varnothing ,i\ne j$
$A^m=\{(i,j)\in A|j\in N^m,i\in N\}$
$\cup \{(j,k)\in A|j\in N^m,k\in N\}$	-	set of all arcs of a subnetwork $g^m$
$I^m=\{(i,j)\in A^m|i,j\in N^m\}$	-	set of internal arcs of the subnetwork $g^m$
$L^m=\left\{l\right|l=e(i,j),(i,j)\in A^m\}$	-	set of all labels of links of a subnetwork $g^m$
$x_w^m$	-	flow rate for the demand w in the subnetwork $g^m$
$b_{wl}^m$	-	binary routing variable for the flow w in the subnetwork $g^m$ and link $l\in L^m$
$E^m=A^m\backslash I^m$	-	set of external arcs (incoming or outgoing)
		of the subnetwork $g^m$
$D^m=\left\{k\right|k\ne m,\exists (i,j)\in E^m:$	-	indices of subnetworks directly connected
$(i\in N^k)\vee (j\in N^k)\}$		with $g^m$
$U^m=\left\{l\right|l=e(i,j),(i,j)\in E^m\}$	-	set of labels of external links for the subnetwork $g^m$
${\xi }_{wl}=\left\{\begin{array}{cc}0.5,\hfill & l\in {\bigcup }_{m=1}^S{U}^m\hfill \\ 1,\hfill & otherwise\hfill \end{array}\right.$	-	correcting coefficient to avoid adding external arcs twice.

Now, the $P_{cl}$ problem can be formulated in the following way:

$$\begin{array}{ccc}\hfill & P_{cl}^{sub}:\hfill & \hfill \underset{x,y,b}{min}\sum _{m=1}^S\sum _{w\in W}\left[\gamma {\left({\overline{x}}_w-x_w^m\right)}^2+\delta \sum _{l\in L_m}{\xi }_{wl}{b}_{wl}^m\right]\end{array}$$

(52)

subject to

$$\begin{array}{cc}\hfill \sum _{\{i\in N|(i,j)\in A^m\}}{b}_{we(i,j)}^m-\sum _{\{k\in N|(j,k)\in A^m\}}{b}_{we(j,k)}^m=& \left\{\begin{array}{cc}-1,\hfill & j=s\left(w\right)\hfill \\ 1,\hfill & j=d\left(w\right)\hfill \\ 0,\hfill & otherwise\hfill \end{array}\right.,\hfill \\ \hfill & \forall w\in W,j\in N^m,m=1,\dots ,S\hfill \end{array}$$

(53)

$$\begin{array}{cccc}\hfill & x_w^m=x_w^n,\hfill & \hfill & \forall w\in W,n\in D^m,m=1,\dots ,S;n>m\hfill \end{array}$$

(54)

$$\begin{array}{cccc}\hfill & b_{wl}^m=b_{wl}^n,\hfill & \hfill & \forall w\in W,l\in U^m,n\in D^m,m=1,\dots ,S;n>m\hfill \end{array}$$

(55)

$$\begin{array}{cccc}\hfill & {\underline{x}}_w\le x_w^m\le {\overline{x}}_w,\hfill & \hfill & \forall w\in W,m=1,\dots ,S\hfill \end{array}$$

(56)

$$\begin{array}{cccc}\hfill & b_{wl}^m\in \{0,1\},\hfill & \hfill & \forall w\in W,\forall l\in L^m,m=1,\cdots ,S\hfill \end{array}$$

(57)

$$\begin{array}{cccc}\hfill & \sum _{w\in W}{y}_{wl}^m\le c_l,\hfill & \hfill & \forall l\in L^m,m=1,\dots ,S\hfill \end{array}$$

(58)

$$\begin{array}{cccc}\hfill & y_{wl}^m\ge x_w^m-{\overline{x}}_w(1-b_{wl}^m),\hfill & \hfill & \forall w\in W,\forall l\in L^m,m=1,\dots ,S\hfill \end{array}$$

(59)

$$\begin{array}{cccc}\hfill & y_{wl}^m\ge 0,\hfill & \hfill & \forall w\in W,\forall l\in L^m,m=1,\dots ,S\hfill \end{array}$$

(60)

$$\begin{array}{cccc}\hfill & y_{wl}^m\le x_w^m,\hfill & \hfill & \forall w\in W,\forall l\in L^m,m=1,\dots ,S\hfill \end{array}$$

(61)

$$\begin{array}{cccc}\hfill & y_{wl}^m\le {\overline{x}}_w{b}_{wl}^m,\hfill & \hfill & \forall w\in W,\forall l\in L^m,m=1,\dots ,S\hfill \end{array}$$

(62)

The $P_{alt}$ problem will obtain the form:

$$\begin{array}{ccc}\hfill & P_{alt}^{sub}:\hfill & \hfill \underset{x,y,b}{min}\sum _{m=1}^S\sum _{w\in W}\left[\gamma {\left({\overline{x}}_w-x_w^m\right)}^2+\delta \sum _{l\in L_m}{\xi }_{wl}{b}_{wl}^m\right]\end{array}$$

(63)

subject to

$$\begin{array}{cc}\hfill \sum _{\{i\in N|(i,j)\in A^m\}}{y}_{we(i,j)}^m-\sum _{\{k\in N|(j,k)\in A^m\}}{y}_{we(j,k)}^m=& \left\{\begin{array}{cc}-x_w^m,\hfill & j=s\left(w\right)\hfill \\ x_w^m,\hfill & j=d\left(w\right)\hfill \\ 0,\hfill & otherwise\hfill \end{array}\right.,\hfill \\ \hfill & \forall w\in W,j\in N^m,m=1,\dots ,S\hfill \end{array}$$

(64)

$$\begin{array}{cccc}\hfill & x_w^m=x_w^n,\hfill & \hfill & \forall w\in W,n\in D^m,m=1,\dots ,S;n>m\hfill \end{array}$$

(65)

$$\begin{array}{cccc}\hfill & b_{wl}^m=b_{wl}^n,\hfill & \hfill & \forall w\in W,l\in U^m,n\in D^m,m=1,\dots ,S;n>m\hfill \end{array}$$

(66)

$$\begin{array}{cccc}\hfill & {\underline{x}}_w\le x_w^m\le {\overline{x}}_w,\hfill & \hfill & \forall w\in W,m=1,\dots ,S\hfill \end{array}$$

(67)

$$\begin{array}{cccc}\hfill & b_{wl}^m\in \{0,1\},\hfill & \hfill & \forall w\in W,\forall l\in L^m,m=1,\cdots ,S\hfill \end{array}$$

(68)

$$\begin{array}{cccc}\hfill & \sum _{(i,j)\in A^m}{b}_{w,e(i,j)}^m\le 1,\hfill & \hfill & \forall w\in W,i\in N^m\hfill \end{array}$$

(69)

$$\begin{array}{cccc}\hfill & \sum _{w\in W}{y}_{wl}^m\le c_l,\hfill & \hfill & \forall l\in L^m,m=1,\dots ,S\hfill \end{array}$$

(70)

$$\begin{array}{cccc}\hfill & y_{wl}^m\ge x_w^m-{\overline{x}}_w(1-b_{wl}^m),\hfill & \hfill & \forall w\in W,\forall l\in L^m,m=1,\dots ,S\hfill \end{array}$$

(71)

$$\begin{array}{cccc}\hfill & y_{wl}^m\ge 0,\hfill & \hfill & \forall w\in W,\forall l\in L^m,m=1,\dots ,S\hfill \end{array}$$

(72)

$$\begin{array}{cccc}\hfill & y_{wl}^m\le x_w^m,\hfill & \hfill & \forall w\in W,\forall l\in L^m,m=1,\dots ,S\hfill \end{array}$$

(73)

$$\begin{array}{cccc}\hfill & y_{wl}^m\le {\overline{x}}_w{b}_{wl}^m,\hfill & \hfill & \forall w\in W,\forall l\in L^m,m=1,\dots ,S\hfill \end{array}$$

(74)

Connections between subnetworks do not allow for solving problems independently for every subnetwork $g^m,m\in 1,..,S$. Internetwork connections correspond to the duplicated flow rates and routing variables equalities (54) and (55) or (65) and (66), so these constraints should be relaxed to achieve decomposable structure.

The Lagrange function for the $P_{cl}^{sub}$ and $P_{alt}^{sub}$ problems is the following:

$$L^{sub}(x,y,b,\alpha ,\beta )=\sum _{m=1}^S\sum _{w\in W}\left[\gamma {\left({\overline{x}}_w-x_w^m\right)}^2+\delta \sum _{l\in L^m}{\xi }_{wl}{b}_{wl}^m\right]$$

$$+\sum _{m=1}^S\sum _{\begin{array}{c}n\in D^m\\ n>m\end{array}}\sum _{w\in W}{\alpha }_w^{mn}\left(x_w^m-x_w^n\right)+\sum _{m=1}^S\sum _{\begin{array}{c}n\in D^m\\ n>m\end{array}}\sum _{w\in W}\sum _{l\in L^m}{\beta }_{wl}^{mn}\left(b_{wl}^m-b_{wl}^n\right)$$

(75)

where ${\alpha }_w^{mn},{\beta }_{wl}^{mn}$ for $l\in L^m,n>m,m=1,\dots ,S,w\in W$ are Lagrange multipliers.

The dual function for the $P_{cl}^{sub}$ problem can be calculated as:

$$L_{Dcl}^{sub}(\alpha ,\beta )=\underset{x,y,b}{min}{L}^{sub}(x,y,b,\alpha ,\beta )$$

(76)

subject to (53), (56)–(62), where the function $L^{sub}(x,y,b,\alpha ,\beta )$ is given by (75).

The dual function for the $P_{alt}^{sub}$ problem can be calculated as:

$$L_{Dalt}^{sub}(\alpha ,\beta )=\underset{x,y,b}{min}{L}^{sub}(x,y,b,\alpha ,\beta )$$

(77)

subject to (64), (67)–(74).

The dual function components can be calculated independently for every subnetwork $g^m,m\in 1,\cdots ,S$ because all constraints representing connections between subnetworks have been relaxed. The number of the relaxed constraints depends on the number of subnetworks S and on the number of connections between them. These constraints have to be relaxed for every demand $w\in W$.

4.3. Continuous and Binary Variables Decomposition

Decomposition presented in this subsection splits the original MINLP problem into two subproblems: the first containing only continuous variables and the second containing only binary variables. Such decomposition can be performed for both problem formulations $P_{cl}$ and $P_{alt}$.

Continuous and binary variables decomposition is achieved by relaxing constraints containing binary and continuous variables simultaneously, that is, constraints (12), (15) for $P_{cl}$ and (23) for $P_{alt}$. The Lagrange function for the $P_{cl}$ problem will be:

$$L_{cl}^{var}(x,y,b,\alpha ,\beta )=\sum _{w\in W}\left[\gamma {\left({\overline{x}}_w-x_w\right)}^2+\delta \sum _{l\in L}{b}_{wl}\right]$$

$$+\sum _{w\in W}\sum _{l\in L}{\alpha }_{wl}\left(x_w-y_{wl}+{\overline{x}}_w{b}_{wl}-{\overline{x}}_w\right)+\sum _{w\in W}\sum _{l\in L}{\beta }_{wl}\left(y_{wl}-{\overline{x}}_w{b}_{wl}\right)$$

(78)

The dual function for the $P_{cl}$ problem with the separation of continuous and binary variables is calculated in the following way:

$$\begin{array}{cc}\hfill & L_{Dcl}^{var}(\alpha ,\beta )=L_{Dcl\_c}^{var}(\alpha ,\beta )+L_{Dcl\_b}^{var}(\alpha ,\beta )-\sum _{w\in W}\sum _{l\in L}{\alpha }_{wl}{\overline{x}}_w\hfill \end{array}$$

(79)

where

$$L_{Dcl\_c}^{var}(\alpha ,\beta )=\underset{x,y}{min}\sum _{w\in W}\left[\gamma {\left({\overline{x}}_w-x_w\right)}^2+\sum _{l\in L}{\alpha }_{wl}{x}_w+\sum _{l\in L}\left({\beta }_{wl}-{\alpha }_{wl}\right)y_{wl}\right]$$

(80)

subject to

$$\begin{array}{ccccc}\hfill & {\underline{x}}_w\le x_w\le {\overline{x}}_w,\hfill & \hfill & \forall w\in W\hfill & \end{array}$$

(81)

$$\begin{array}{cccc}\hfill & \sum _{w\in W}{y}_{wl}\le c_l,\hfill & \hfill & \forall l\in L\hfill \end{array}$$

(82)

$$\begin{array}{cccc}\hfill & y_{wl}\ge 0,\hfill & \hfill & \forall w\in W,\forall l\in L\hfill \end{array}$$

(83)

$$\begin{array}{cccc}\hfill & y_{wl}\le x_w,\hfill & \hfill & \forall w\in W,\forall l\in L\hfill \end{array}$$

(84)

and

$$L_{Dcl\_b}^{var}(\alpha ,\beta )=\underset{b}{min}\sum _{w\in W}\sum _{l\in L}\left[\delta b_{wl}+\left({\alpha }_{wl}-{\beta }_{wl}\right){\overline{x}}_w{b}_{wl}\right]$$

(85)

subject to

$$\begin{array}{ccc}\hfill \forall w\in W,& \sum _{\{i\in N|(i,j)\in A\}}{b}_{w,e(i,j)}-\sum _{\{k\in N|(j,k)\in A\}}{b}_{w,e(j,k)}=\hfill & \hfill \left\{\begin{array}{cc}-1,\hfill & j=s\left(w\right)\hfill \\ 1,\hfill & j=d\left(w\right)\hfill \\ 0,\hfill & otherwise\hfill \end{array}\right.\end{array}$$

(86)

After decomposition, the dual function consists of two parts: the first $L_{Dcl\_c}$ contains continuous variables, the second $L_{Dcl\_b}$ contains binary variables. The binary part of the dual function consists of the linear objective function and network flow conservation law rule (86). These constraints can be presented by a unimodular matrix. As a result, $L_{Dcl\_b}$ has an integrality property, that is, the problem decision variables at the optimum are always integer. Thus, the constraints (10) preserving that the route variables are binary can be safely omitted. Hence, the decomposed problem does not operate on the mixed domain.

Unfortunately, the number of relaxed constraints is rather big—$2\left|L\right|\left|W\right|$. It may result in unsatisfactory tightness of the relaxation.

The Lagrange function for the $P_{alt}$ problem with a separation of continuous and binary variables is:

$$\begin{array}{cc}\hfill L_{alt}^{var}(x,y,b,\beta )=& \sum _{w\in W}\left[\gamma {\left({\overline{x}}_w-x_w\right)}^2+\delta \sum _{l\in L}{b}_{wl}\right]+\sum _{w\in W}\sum _{l\in L}{\beta }_{wl}\left(y_{wl}-{\overline{x}}_w{b}_{wl}\right)\hfill \end{array}$$

(87)

The dual function for the $P_{alt}$ problem with the separation of continuous and binary variables is following:

$$L_{Dalt}^{var}\left(\beta \right)=L_{Dalt\_c}^{var}\left(\beta \right)+L_{Dalt\_b}^{var}\left(\beta \right)$$

(88)

where

$$L_{Dalt\_c}^{var}\left(\beta \right)=\underset{x,y}{min}\sum _{w\in W}\left[\gamma {\left({\overline{x}}_w-x_w\right)}^2+\sum _{l\in L}{\beta }_{wl}{y}_{wl}\right]$$

(89)

subject to

$$\forall w\in W,\sum _{\{i\in N|(i,j)\in A\}}{y}_{w,e(i,j)}-\sum _{\{k\in N|(j,k)\in A\}}{y}_{w,e(j,k)}=\left\{\begin{array}{cc}-x_w,\hfill & j=s\left(w\right)\hfill \\ x_w,\hfill & j=d\left(w\right)\hfill \\ 0,\hfill & otherwise\hfill \end{array}\right.$$

(90)

$$\begin{array}{cccc}\hfill & {\underline{x}}_w\le x_w\le {\overline{x}}_w,\hfill & \hfill & \forall w\in W\hfill \end{array}$$

(91)

$$\begin{array}{cccc}\hfill & \sum _{w\in W}{y}_{wl}\le c_l,\hfill & \hfill & \forall l\in L\hfill \end{array}$$

(92)

$$\begin{array}{cccc}\hfill & y_{wl}\ge 0,\hfill & \hfill & \forall w\in W,\forall l\in L\hfill \end{array}$$

(93)

and

$$L_{Dalt\_b}^{var}\left(\beta \right)=\underset{b}{min}\sum _{w\in W}\sum _{l\in L}\left(\delta b_{wl}-{\beta }_{wl}{\overline{x}}_w{b}_{wl}\right)$$

(94)

subject to

$$\begin{array}{cccc}\hfill & \sum _{\{j\in N|(i,j)\in A\}}{b}_{w,e(i,j)}\le 1,\hfill & \hfill & \forall w\in W,i\in N\hfill \end{array}$$

(95)

$$\begin{array}{cccc}\hfill & b_{wl}\in \{0,1\},\hfill & \hfill & \forall w\in W,\forall l\in L\hfill \end{array}$$

(96)

For the alternative problem formulation (16)–(23), continuous and binary variable decomposition requires the relaxation of constraints (23). The number of the relaxed constraints in this case is equal to $\left|W\right|\left|L\right|$. However, the binary subproblem does not possess integrality property in this case. In addition, violations of the relaxed constraints (23) violate the relation between routing variables $b_{wl}$ and auxiliary variables $y_{wl}$, which can result in the violations of network flow balance constraints (2). Thus, the following conclusions can be made: decomposition of $P_{alt}$ allows for getting tighter bounds than decomposition of $P_{cl}$, but subproblems of the decomposed $P_{alt}$ are harder to solve than subproblems of the decomposed $P_{cl}$.

5. Coordination Algorithms

A dual problem is solved iteratively. In every iteration for given Lagrange multipliers, a dual function is computed (local problem) and Lagrange multipliers are updated (the coordination problem). Several coordination algorithms were used to solve the dual problems presented in the previous section.

1.

Simple gradient algorithms

A simple gradient algorithm to update Lagrange multipliers directly uses the gradient of the dual function, in the following manner:

• For multipliers of the relaxed equality constraints:

  $${\lambda }_{k+1}={\lambda }_k+{\eta }_k\nabla L_D\left({\lambda }_k\right);$$

  (97)
• For multipliers of the relaxed inequality constraints:

  $${\lambda }_{k+1}=max\left(0,{\lambda }_k+{\eta }_k\nabla L_D\left({\lambda }_k\right)\right).$$

  (98)

Parameter ${\eta }_k$ (k - iteration number) plays a key role in the simple gradient coordination algorithm. It is responsible for the algorithm convergence rate. Some simple methods for ${\eta }_k$ parameter computation can be used [45]:

• Square summable, but not summable:

  $$\sum _{k=1}^{\infty }{\eta }_k^2<\infty ,\sum _{k=1}^{\infty }{\eta }_k=\infty$$

  (99)

  $${\eta }_k=\frac{p}{r+k},p>0,r\ge 0$$

  (100)
• Nonsummable diminishing:

  $$\underset{k\to \infty }{lim}{\eta }_k=0,\sum _{k=1}^{\infty }{\eta }_k=\infty$$

  (101)

  $${\eta }_k=\frac{p}{\sqrt{k}},p>0$$

  (102)

More sophisticated techniques for ${\eta }_k$ evaluation can also be used. Some of them utilize knowledge of the optimal objective value (or its estimate). One of the most advanced methods is Goffin and Kiwiel’s algorithm of the simultaneous optimal value estimation and Lagrange multipliers update [46], which guarantees convergence with an acceptable rate and relatively easy tuning of parameters. This very version of the simple gradient coordination algorithm has been used in our numerical tests.

The simple gradient algorithm is strongly dependent on its parameters. Quite often, its convergence rate can be unsatisfactory. However, this algorithm is relatively easy to implement and requires rather small computational efforts. Hence, for a large number of the Lagrange multipliers, this coordination method may be the most appropriate.

2.

Cutting plane method

Cutting plane algorithms also utilize dual function gradients, but in a different way than simple gradient algorithms. Namely, the approximation of the dual function is based on the dual function gradient in the following manner (for simplicity, we assume that the problem has only inequality constraints):

$$\begin{array}{c}\hfill L_{D_a}^k\left(\lambda \right)=min\left\{L_D\left({\lambda }_0\right)+{(\lambda -{\lambda }_0)}^T\nabla L_D\left({\lambda }_0\right),...,L_D\left({\lambda }_{k-1}\right)+{(\lambda -{\lambda }_{k-1})}^T\nabla L_D\left({\lambda }_{k-1}\right)\right\}\end{array}$$

(103)

The approximated dual problem

$$\begin{array}{c}\hfill D_a^k=\underset{\lambda \ge 0}{max}{L}_{D_a}^k\left(\lambda \right)\end{array}$$

(104)

can be reformulated as:

$$\begin{array}{cc}\hfill & D_a^k=\underset{z,\lambda \ge 0}{max}z\hfill \end{array}$$

(105)

$$\begin{array}{ccc}\hfill & z\le L_D\left({\lambda }_0\right)+{(\lambda -{\lambda }_0)}^T\nabla L_D\left({\lambda }_0\right)\hfill & \hfill \\ \hfill & ⋮\hfill \end{array}$$

(106)

$$\begin{array}{cc}\hfill & \hspace{1em}\hspace{1em}z\le L_D\left({\lambda }_{k-1}\right)+{(\lambda -{\lambda }_{k-1})}^T\nabla L_D\left({\lambda }_{k-1}\right)\hfill \end{array}$$

(107)

In the cutting plane algorithm, dual function approximation $L_{D_a}^k$ is built iteratively. New cuts, defined by linear functions which use dual function gradients, are added in every iteration. The solution of the approximated dual problem is a point where the new cut will be put. It means that, with every iteration, a better piecewise linear approximation of the dual function is achieved. And then, in every iteration of the algorithm, the optimization problem described by (105)–(107) is solved.

A similar, but more sophisticated algorithm (which will be used in numerical tests) can also be applied to approximate the dual function and to solve the dual problem [47]. Namely, the dual function can be formulated in the following manner:

$$\begin{array}{c}\hfill D_a^k=\underset{\lambda \ge 0}{max}\left[z-\frac{1}{2}{\parallel \lambda -{\overline{\lambda }}_k\parallel }^2\right]\end{array}$$

(108)

In the case of the (108) formula, a penalty component was added to keep a stable convergence rate of the algorithm. Two kinds of steps are described in [47] for the algorithm. The first is a significant step: ${\overline{\lambda }}_{k+1}={\lambda }_{k+1}$, the second is zero-step: ${\overline{\lambda }}_{k+1}={\lambda }_k$. Zero step still uses new Lagrange multipliers to build a better approximation, but does not update a penalty parameter, so it allows for achieving a better approximation. Meanwhile, a significant step requires significant reduction of the dual value and is performed when:

$$L_D\left({\lambda }_{k+1}\right)-L_D\left({\lambda }_k\right)\le m\left[L_{D_a}^k\left({\lambda }_{k+1}\right)-L_D\left({\lambda }_k\right)\right]$$

(109)

where parameter $m\in (0,1)$. The problem of the inactive cuts gathering is also solved in [47]. They can be safely removed from the dual problem formulation when Lagrange multipliers of inactive cuts are equal to zero. Thus, after solving the approximated dual problem (108) in every iteration, Lagrange multipliers of the cuts (106)–(107) are retrieved and only cuts with nonzero Lagrange multipliers are kept; those remaining are removed.

To summarize, cutting plane methods are more complicated and require more computational effort for solving an additional optimization problem in every iteration. The problem considered in this paper may become rather complicated for a greater number of Lagrange multipliers. Fortunately, this method allows for achieving a more precise result and is almost independent of parameters tuning.

6. Numerical Tests

The methods described above were studied and tested on several problems. Networks of different sizes were generated with the sets of demands defined for them. Then, the problems of simultaneous routing and flow allocation were formulated for these networks. The tested networks consist of multiple nodes clusters, which are loosely connected. However, nodes inside the clusters are strongly connected. Such network configuration allows for performing subnetworks decomposition relaxing a relatively small number of constraints.

First, problems P were solved by an MINLP solver to obtain optimal solutions. Then, decomposition methods based on Lagrangian relaxation were applied to them. Multiple local problems obtained from decompositions were solved sequentially (can be solved in parallel), and the selected coordination algorithm updated Lagrange multipliers. The process of solving local problems and updating Lagrange multipliers for them was performed iteratively until a stop condition was met. All combinations of the Lagrangian relaxation based decomposition methods and coordination algorithms were studied.

Finally, a simple heuristic method, namely a Heuristic Routes Finding (HRF) algorithm presented below, was applied to solve these optimization problems in a decomposed manner.

All implementations and numerical tests were performed using a Python programming language. Pyomo software package was used to formulate and analyze optimization problems. The following optimization software packages were used: CPLEX (local problems solver, primal problem solver, R problem of the heuristic algorithm solver) and IPOPT (coordination problem solver). A NetworkX software package was used to generate and visualize optimization problems’ networks.

6.1. Simple Heuristic Routes Finding (HRF) Algorithm

Heuristic methods do not guarantee achieving the optimal solution. However, such methods may allow solving problems at lower costs. Article [48] describes and improves a certain heuristic method for solving the optimization problem of the simultaneous routing and flow bandwidth allocation. The heuristic approach presented there separates the routing problem and flow rate allocation. First, the weight of every arc is calculated as the minimum of capacity for a given link and flow rate upper bound to prevent flow rates’ upper bound violations. Then, the Dijkstra shortest path algorithm is applied to every demand w, that is, for every pair of source $s\left(w\right)$ and destination $d\left(w\right)$ nodes, the optimal path $b_w^{*}$ is calculated. In the second step, for known optimal paths (routes) $b^{*}$, the following flow allocation problem is solved as a continuous nonlinear programming problem:

$$\begin{array}{cc}\hfill R:& \underset{x}{min}\sum _{w\in W}\gamma {\left({\overline{x}}_w-x_w\right)}^2\hfill \end{array}$$

(110)

subject to

$$\begin{array}{ccc}{\underline{x}}_w\le x_w\le {\overline{x}}_w,\hfill & \hfill & \forall w\in W\hfill \end{array}$$

(111)

$$\begin{array}{cccc}\hfill & {\displaystyle \sum _{w\in W}{b}_{wl}^{*}{x}_w\le c_l,}\hfill & \hfill & \forall l\in L\hfill \end{array}$$

(112)

6.2. Generation of Networks

For numerical tests, two network topologies were generated: the first with a lower number of variables (referred to as a medium network)—see

Table 1. Medium problem instances’ parameters.

Nodes Number	Arcs Number	Demands Number	Flow Rates Bound	Capacities Bound
25	82	12	[0.001, 3]	[0.3, 1]

and Figure 1 and the second with a greater number of variables (referred to as a large network)—see

Table 2. Large problem instances’ parameters.

Nodes Number	Arcs Number	Demands Number	Flow Rates Bound	Capacities Bound
49	146	32	[0.001, 3]	[0.3, 1]

and Figure 2. Five seeds were used to formulate randomized problem instances. For every problem instance, the source and the destination nodes were generated randomly, and so were the capacities of links. Problem instances were generated for both network topologies. Several approaches to optimization problem formulation were used. Networks and their parameters, the results of network decomposition achieved with HRF heuristics and the results achieved with several Lagrange decomposition methods (in combination with two coordination algorithms) will be presented in this section.

6.3. Comparison of Results for Different Methods

The numerical tests were made only for formulations $P_{cl}$ and $P_{alt}$ (which differ in flow conservation equations using, respectively, binary or continuous variables) because the only nonlinear component in these formulations is a convex quadratic part of the objective function. Meanwhile, the formulation P has nonconvex constraints, which complicate solving the problem. All three decomposition approaches that is: demand, subnetworks and binary/continuous variables were checked.

Numerical experiments were performed according to the following scenario: first, the CPLEX solver attempted to solve all problem formulations; then, all decomposition methods were applied; finally, a feasible solution was restored if it was possible.

The following items are used to describe the results of computations:

Objective	—value of the problem objective function,
Spent Time	—time spent to retrieve solutions in seconds,
Feasibility	—solution feasibility validation (for the original problem P); for infeasible solution number of capacity constraints (5) violations—“CCV: <number>”, and number of routing constraints (2) violations
	—“RCV: <number>”,
Status	—solution status retrieved from a solver,
Dual Objective (LB)	—value of the dual problem objective function, which is also a lower bound for the primal problem,
Heuristic Objective (LB)	—value of the problems objective function retrieved using the HRF Algorithm,
Reallocated (UB)	—value of a feasible solution restored from a lower bound solution routing variables using solution of the R problem, which is also an upper bound for the primal problem,
LB Relative Gap	—ratio of the lower bound gap with respect to the optimal objective given in percents ($\frac{Objective-LB}{Objective}·100\%$)
UB Relative Gap	—ratio of the upper bound gap with respect to the optimal objective given in percents ($\frac{UB-Objective}{Objective}·100\%$)

The tests were performed on the machine with Intel Core i7-4710 HQ, 4 core 2.5–3.5 GHz processor, 16 GB of DDR3-1600, dual-channel memory and 512 GB SSD storage, under Ubuntu 18.04 operating system. As it was said before, every numerical test was executed for five randomly generated problem instances.

The results of optimization obtained by CPLEX solver for medium-sized networks are presented in

Table 3. Numerical results for medium problems—CPLEX solver.

	Objective	Spent Time [s]	Feasibility	Status
Problem $P_{cl}$
1	89.62	4.39	Feasible	Optimal
2	89.64	34.61	Feasible	Optimal
3	83.68	273.25	Feasible	Optimal
4	84.35	271.22	Feasible	Optimal
5	85.16	725.94	Feasible	Optimal
Problem $P_{alt}$
1	89.62	12.16	Feasible	Optimal
2	89.64	825.28	Feasible	Optimal
3	83.68	301.64	Feasible	Optimal
4	84.35	348.25	Feasible	Optimal
5	85.16	2000.09	Feasible	Timeout

. These results will serve further as a benchmark for us. In particular, the gaps are calculated in relation to them.

The average results of tests of different decomposition approaches are presented in Figure 3, Figure 4 and Figure 5; the detailed results for all medium-sized networks in

Table 4. Numerical results for medium problems—demands decomposition.

	Dual Objective (LB)	Reallocated (UB)	Spent Time [s]	Feasibility	LB Relative Gap [%]	UB Relative Gap [%]
Formulation $P_{cl}$ Coordination: Proximal Cutting Planes
1	86.02	92.9	21.63	CCV: 14; RCV: 0	4.0170	3.6599
2	87.22	101.33	27.86	CCV: 19; RCV: 0	2.6997	13.0411
3	77.17	84.76	26.28	CCV: 8; RCV: 0	7.7796	1.2906
4	80.82	90.16	21.28	CCV: 13; RCV: 0	4.1849	6.8880
5	83.04	91.61	26.04	CCV: 13; RCV: 0	2.4894	7.5740
Formulation $P_{alt}$ Coordination: Proximal Cutting Planes
1	86.5	93.15	19.49	CCV: 13; RCV: 0	3.4814	3.9389
2	87.31	94.41	22.24	CCV: 19; RCV: 0	2.5993	5.3213
3	77.19	87.3	23.29	CCV: 10; RCV: 0	7.7557	4.3260
4	80.52	87.19	25.07	CCV: 8; RCV: 0	4.5406	3.3669
5	82.95	90.13	22.87	CCV: 11; RCV: 0	2.5951	5.8361
Formulation $P_{cl}$ Coordination: Simple Gradient Level Method
1	83.14	91.23	19.77	CCV: 15; RCV: 0	7.2305	1.7965
2	84.62	94.75	29.56	CCV: 19; RCV: 0	5.6002	5.7006
3	74.52	94.37	27.19	CCV: 14; RCV: 0	10.9465	12.7749
4	78.14	86.57	30.81	CCV: 13; RCV: 0	7.3622	2.6319
5	79.49	87.9	27.82	CCV: 19; RCV: 0	6.6581	3.2175
Formulation $P_{alt}$ Coordination: Simple Gradient Level Method
1	84.42	92.64	16.72	CCV: 14; RCV: 0	5.8023	3.3698
2	84.83	95	18.23	CCV: 20; RCV: 0	5.3659	5.9795
3	74.97	90.07	19.66	CCV: 10; RCV: 0	10.4087	7.6362
4	76.96	86.86	19.12	CCV: 18; RCV: 0	8.7611	2.9757
5	79.52	92.72	17.32	CCV: 19; RCV: 0	6.6228	8.8774

,

Table 5. Numerical results for medium problems—subnetworks decomposition.

	Dual Objective (LB)	Spent Time [s]	Feasibility	LB Relative Gap [%]
Formulation $P_{cl}$ Coordination: Proximal Cutting Planes
1	88.76	37.34	CCV: 18; RCV: 14	0.9596
2	86.26	770.63	CCV: 29; RCV: 24	3.7706
3	80.51	4748.93	CCV: 24; RCV: 18	3.7882
4	80.21	82.85	CCV: 18; RCV: 20	4.9081
5	81.63	3767.56	CCV: 25; RCV: 31	4.1451
Formulation $P_{alt}$ Coordination: Proximal Cutting Planes
1	88.9	40.87	CCV: 10; RCV: 18	0.8034
2	84.82	132.7	CCV: 20; RCV: 31	5.3771
3	80	221.48	CCV: 14; RCV: 26	4.3977
4	80.7	111.24	CCV: 15; RCV: 30	4.3272
5	82.25	328.37	CCV: 17; RCV: 34	3.4171
Formulation $P_{cl}$ Coordination: Simple Gradient Level Method
1	88.13	35.6	CCV: 17; RCV: 4	1.6626
2	85.39	524.38	CCV: 26; RCV: 14	4.7412
3	79.26	2072.34	CCV: 25; RCV: 10	5.2820
4	80.08	51.44	CCV: 30; RCV: 16	5.0622
5	79.88	597.26	CCV: 30; RCV: 18	6.2001
Formulation $P_{alt}$ Coordination: Simple Gradient Level Method
1	87.02	25.7	CCV: 18; RCV: 14	2.9011
2	83.48	202.28	CCV: 23; RCV: 18	6.8719
3	77.94	215.1	CCV: 22; RCV: 23	6.8595
4	80.1	106.3	CCV: 17; RCV: 26	5.0385
5	78.83	91.08	CCV: 22; RCV: 28	7.4331

,

Table 6. Numerical results for medium problems—continuous and binary variables decomposition.

	Dual Objective (LB)	Reallocated (UB)	Spent Time [s]	Feasibility	LB Relative Gap [%]	UB Relative Gap [%]
Formulation $P_{cl}$ Coordination: Proximal Cutting Planes
1	63.38	90.93	14.32	CCV: 19; RCV: 0	29.2792	1.4617
2	65.05	97.68	14.17	CCV: 18; RCV: 0	27.4320	8.9692
3	36.08	89.55	15.04	CCV: 19; RCV: 0	56.8834	7.0148
4	38.75	87.69	15.73	CCV: 29; RCV: 0	54.0605	3.9597
5	34.46	92.54	16.38	CCV: 32; RCV: 0	59.5350	8.6660
Formulation $P_{alt}$ Coordination: Proximal Cutting Planes
1	85.61	-	14.97	CCV: 0; RCV: 24	4.4744	-
2	85.12	-	14.11	CCV: 0; RCV: 24	5.0424	-
3	75.35	-	13.06	CCV: 0; RCV: 24	9.9546	-
4	79.31	-	14.51	CCV: 0; RCV: 24	5.9751	-
5	81.44	-	15.03	CCV: 0; RCV: 24	4.3682	-
Formulation $P_{cl}$ Coordination: Simple Gradient Level Method
1	53.53	97.07	4.04	CCV: 30; RCV: 0	40.2700	8.3129
2	48.27	98.97	4.09	CCV: 41; RCV: 0	46.1513	10.4083
3	24.6	90.62	4.18	CCV: 32; RCV: 0	70.6023	8.2935
4	29.43	92.31	4.33	CCV: 38; RCV: 0	65.1097	9.4369
5	20.5	94.11	4.27	CCV: 39; RCV: 0	75.9277	10.5096
Formulation $P_{alt}$ Coordination: Simple Gradient Level Method
1	85.79	-	1.86	CCV: 3; RCV: 24	4.2736	-
2	85.34	-	2.02	CCV: 3; RCV: 26	4.7970	-
3	75.55	-	1.87	CCV: 0; RCV: 24	9.7156	-
4	79.51	-	2.13	CCV: 2; RCV: 22	5.7380	-
5	81.7	-	1.95	CCV: 1; RCV: 30	4.0629	-

and

Table 7. Numerical results for medium problems—Dijkstra algorithm based heuristics (HRF).

	Heuristic Objective (LB)	Reallocated (UB)	Spent Time [s]	Feasibility	LB Relative Gap [%]	UB Relative Gap [%]
Dijkstra algorithm based heuristics (HRF)
1	73.1	96.06	4.86	CCV: 14; RCV: 0	18.4334	7.1859
2	71.41	103.29	4.76	CCV: 14; RCV: 0	20.3369	15.2276
3	69.65	94.56	4.48	CCV: 15; RCV: 0	16.7663	13.0019
4	70.35	94.79	3.66	CCV: 16; RCV: 0	16.5975	12.3770
5	73.47	97.44	5.37	CCV: 13; RCV: 0	13.7271	14.4199

.

The demands decomposition method (see Table 4) solved almost all medium problem instances in a shorter time than CPLEX. Solutions obtained from this method provide problem lower bounds—relative gaps of these solutions are within the range: 3–11%. Unfortunately, all retrieved solutions violated capacity constraints. Feasible solutions (upper bounds) obtained from the lower bounds’ routings are within the range: 3–13%. Importantly, tighter lower bounds did not always provide tighter upper bounds.

Subnetworks decomposition (see Table 5) spent an unacceptable amount of time to find solutions for almost all medium-sized problem instances (decomposed problem was solved much longer than the primal problem). Spent time of $P_{alt}$ formulations is much shorter than that of $P_{cl}$ formulations, but still unsatisfactory. Lower bounds provided by subnetworks decomposition are the tightest: 1–7.5%. Unfortunately, all solutions retrieved from this decomposition violated routing constraints. Because of that, as well as because of unacceptable spent times, subnetwork decomposition was not applied to large problem instances.

The continuous and binary variables decomposition method (see Table 6) was the quickest from the Lagrangian based decomposition methods. This decomposition method shows how a large number of Lagrange multipliers increases coordination time for cutting planes’ algorithms. As it can be seen, it provides a big duality gap for the formulation $P_{cl}$. However, the duality gap for this method is small when formulation $P_{alt}$ is used. Unfortunately, such gap is achieved for zero-valued Lagrange multipliers, which means that the bound was not improved (see Figure 6 and Figure 7) by coordination algorithms (zero-valued Lagrange multiplier are the start point of the coordination).

In addition, feasible solutions (upper bounds) were restored from routings obtained from continuous and binary variable decomposition for formulation $P_{cl}$. The tightness of these solutions is comparable to the demand decomposition solutions, regardless of the large duality gap.

The undesirable properties of continuous and binary variable decomposition method are derived from a large number of relaxed constraints—comparable to the number of variables. Because of such unsatisfactory results for medium networks, the continuous and binary variables decomposition method was not used in further numerical tests.

The heuristic method (see Table 7) was not able to provide neither tight lower bounds nor tight upper bounds. Times spent by this method were very small, but still not small enough to use this method in combination with the branch-and-bound algorithm.

From now on, the results of tests for large problems will be presented.

From

Table 8. Numerical results for large problems—CPLEX solver.

	Objective	Spent Time [s]	Feasibility	Status
Problem $P_{cl}$
1	247.08	478.79	Feasible	Optimal
2	239.79	1111.43	Feasible	Optimal
3	246.62	2000.15	Feasible	Timeout
4	248.7	2000.08	Feasible	Timeout
5	252.98	2000.12	Feasible	Timeout
Problem $P_{alt}$
1	247.08	2000.6	Feasible	Timeout
2	239.79	2000.26	Feasible	Timeout
3	246.62	2000.55	Feasible	Timeout
4	248.66	2000.47	Feasible	Timeout
5	252.98	2000.34	Feasible	Timeout

, it is seen that the commercial solver CPLEX was able to find the optimal solutions for only 2 of 10 instances in a time less than 2000 s.

As it was stated before, only a demands decomposition approach was applied and for comparison of the simple HRF method. The average results are presented in Figure 8, Figure 9 and Figure 10, and the detailed ones in

Table 9. Numerical results for large problems—demands decomposition.

	Dual Objective (LB)	Reallocated (UB)	Spent Time [s]	Feasibility	LB Relative Gap [%]	UB Relative Gap [%]
Problem $P_{cl}$ Coordination: Proximal Cutting Planes
1	239.03	250.01	154.07	CCV: 17; RCV: 0	3.2581	1.1859
2	233.17	239.79	130.02	CCV: 17; RCV: 0	2.7607	0.0000
3	241.33	250.82	137.3	CCV: 15; RCV: 0	2.1450	1.7030
4	234.93	252.54	205.87	CCV: 14; RCV: 0	5.5368	1.5440
5	250.49	257.5	141.15	CCV: 13; RCV: 0	0.9843	1.7867
Problem $P_{alt}$ Coordination: Proximal Cutting Planes
1	238.97	250.28	345.64	CCV: 16; RCV: 0	3.2823	1.2951
2	233.11	242.78	201.82	CCV: 14; RCV: 0	2.7858	1.2469
3	241.35	249.96	204.16	CCV: 14; RCV: 0	2.1369	1.3543
4	234.99	252.42	230.01	CCV: 17; RCV: 0	5.4975	1.4958
5	249	255.32	225.91	CCV: 15; RCV: 0	1.5732	0.9250
Problem $P_{cl}$ Coordination: Simple Gradient Level Method
1	234.52	250.47	111.44	CCV: 30; RCV: 0	5.0834	1.3720
2	230.12	246.14	176.46	CCV: 25; RCV: 0	4.0327	2.6482
3	239.22	248.44	117.68	CCV: 21; RCV: 0	3.0006	0.7380
4	230.37	251.91	132.05	CCV: 22; RCV: 0	7.3703	1.2907
5	245.62	257.05	120.96	CCV: 15; RCV: 0	2.9093	1.6088
Problem $P_{alt}$ Coordination: Simple Gradient Level Method
1	233.88	251.28	148.46	CCV: 30; RCV: 0	5.3424	1.6999
2	231.22	242.2	182.3	CCV: 24; RCV: 0	3.5740	1.0050
3	239.59	248.88	185.31	CCV: 19; RCV: 0	2.8505	0.9164
4	231.37	254.85	208.15	CCV: 19; RCV: 0	6.9533	2.4729
5	245.86	253.67	233.64	CCV: 36; RCV: 0	2.8145	0.2727

and

Table 10. Numerical results for large problems—Dijkstra algorithm based heuristics (HRF).

	Heuristic Objective (LB)	Reallocated (UB)	Spent Time [s]	Feasibility	LB Relative Gap [%]	UB Relative Gap [%]
Dijkstra algorithm based heuristics (HRF)
1	199.48	251.52	67.61	CCV: 28; RCV: 0	19.265	1.7970
2	203.27	253.23	84.56	CCV: 40; RCV: 0	15.2300	5.6049
3	190.52	255.81	77.33	CCV: 32; RCV: 0	22.7475	3.7264
4	200.47	256.75	78.81	CCV: 32; RCV: 0	19.3928	3.2368
5	202.66	264.61	91.64	CCV: 32; RCV: 0	19.8909	4.5972

.

Demands decomposition for large problem instances (see Table 9) was able to provide tighter lower bounds (1–7.4%) than for medium problem instances. Unexpectedly, the obtained upper bounds (0–3%) were also much better than for medium problem instances. Since, as it was stated before, the commercial solver CPLEX failed to find the optimal solutions for most of the instances in half an hour, while the demands decomposition method found solutions in few minutes, it is a very good algorithm for real-time systems.

In the case of large problem instances (see Table 10), lower bound relative gaps for solutions found by the heuristic method are comparable to the gaps for medium problem instances. However, the obtained upper bounds gaps were smaller than for medium problem instances, just as with the demands decomposition. Spent times of the heuristic methods were even shorter than for the demand decomposition.

7. Conclusions

The decomposition methods presented in the paper can approximately solve the problems of simultaneous routing and bandwidth allocation in energy-aware networks with less time expenditure than a commercial solver. The quality of the produced solution is strongly dependent on the problem parameters such as: network topology, number of demands, capacities of links, and flow rate bounds. The selection of the appropriate decomposition method should be connected with finding the most scalable component of the solved problems.

The heuristic method presented in Section 6.1 was the fastest one, but the optimal solution approximation provided by it was not precise enough. Because of that, an attempt to combine this method with a branch and bound algorithm was made. Unfortunately, for large problem instances, this approach failed.

The decomposition methods based on Lagrangian relaxation showed a better precision. As it turned out, their performance may strongly depend on the optimization problem formulation. The quality of the solution is better for decomposition methods with a lower number of relaxed constraints. Because of that, the continuous and binary variables decomposition method did not present well.

The subnetworks decomposition can produce quite a good approximation of the optimal value of the primal problem. Unfortunately, this method did not allow for gaining a satisfactory performance boost for the tested instances—finding local problems solutions could be enormously long. Local problems retrieved from the demand decomposition or the subnetworks decomposition are problems of a smaller size, but still mixed-integer and NP-hard, so their complexity may remain significant. Moreover, the solutions delivered by this method do not form paths, and it is impossible to restore a feasible solution in a simple way.

Meanwhile, the demand decomposition method allowed for retrieving a solution with a relatively small duality gap and made this in a much shorter time than a commercial solver. In addition, in the case of demand decomposition, a local problem solution can be used to restore a solution feasible for the primal problem.

Both coordination algorithms used in tests were able to find the solution of the dual problems. Overall, a cutting plane method with the proximal component provided better solutions, but the simple gradient level method was able to produce comparable solutions in a shorter time. Moreover, cutting planes’ methods need much more time with the growth of the number of Lagrange multipliers, so, for problems involving a large number of multipliers, simple gradient algorithms may deliver a significant performance increase.

Despite the usage of the presented methods for finding the exact solution of the primal problem seems to be not promising, the Lagrangian relaxation decomposition methods for large problem instances are still able to provide a good estimate of the optimal value of the objective function in a much shorter time than a commercial solver finds the optimal solution.