A Matching-Strategy-Inspired Preconditioning for Elliptic Optimal Control Problems

Abstract

In this paper, a new preconditioning method is proposed for the linear system arising from the elliptic optimal control problem. It is based on row permutations of the linear system and approximations of the corresponding Schur complement inspired by the matching strategy. The eigenvalue bounds of the preconditioned matrices are shown to be independent of mesh size and regularization parameter. Numerical results illustrate the efficiency of the proposed preconditioning methods.

1. Introduction

The efficient solution of the PDE-constrained optimization problem is of great importance in many areas of science and engineering. The performance of iterative methods for these problems depends highly on the choice of preconditioners, which are drawing more and more attention in various fields [1,2,3,4,5,6,7,8,9,10]. Extensive attention has been drawn to preconditioning the saddle point problem arising from the PDE-constrained optimization problem with different PDE constraints, boundary conditions and control types. As a comparison of preconditioned Krylov subspace iteration methods, Axelsson, Farouq and Neytcheva constructed a series of block matrix preconditioners for optimal control problems with Poisson and convection-diffusion control, respectively [11]. In addition, other preconditioning techniques based on domain decomposition and norm equivalence (see, e.g., Arioli, Kourounis and Loghin [12]) and operator methods (see, e.g., Elvetun and Nielsen [13], Gergelits, Mardal, Nielsen et al. [14]) have also been investigated to help to solve the PDE-constrained optimization problem more efficiently.

In this paper, we consider building block preconditioners for the elliptic optimal control problem

$$\underset{y,u}{\min }\frac{1}{2}\left|\right|y-y_d{\left|\right|}_{L_2\left(\mathsf{\Omega }\right)}^2+\frac{\beta }{2}{\left|\right|u\left|\right|}_{L_2\left(\mathsf{\Omega }\right)}^2,$$

(1)

subject to

$$\left\{\begin{array}{cc}\hfill -{\nabla }^{2}y=u& \mathrm{in}\mathsf{\Omega },\hfill \\ \hfill y=g& \mathrm{on}\partial \mathsf{\Omega },\hfill \end{array}\right.$$

(2)

where $\mathsf{\Omega }\subset {\mathbb{R}}^2$ with boundary $\partial \mathsf{\Omega }$, y is the state variable, u is the control variable, $\beta >0$ is a regularization parameter and $g\in H^1\left(\mathsf{\Omega }\right)$ is some given function. In this problem, the state variable y is supposed to approach the desired state $y_d$ as close as possible under the given constraints. It can be verified that the existence and uniqueness conditions are satisfied in the considered problem. The state Equation (2) has a unique weak solution (see [15,16,17,18] for more details). The optimal control problems (1) and (2) have a unique optimal solution (see [19] for more details). The motivation of this type of problem is to attempt to make the state variable as close as possible to a desired state, while also penalizing the input of a large control into the system. This can be regarded as penalizing the input of energy into a physical system. The effective solving of this type of problem plays an important role in many real applications [20].

Over the last decade, several preconditioning methods have been proposed for the linear system arising from this problem. In [6], Rees et al. approximated the Schur complement by dropping one of its terms and used this strategy to build block preconditioners. In [7], Pearson and Wathen built preconditioners for the same problem based on a matching strategy of the corresponding Schur complement. The main contribution of this paper is to construct new preconditioners for the linear system arising from the elliptic optimal control problem. Our preconditioner will take advantage of the structure of the saddle point problem. We choose to work with a permuted version. This along with the use of the matching strategy yields a different sparse approximation for the Schur complement. The resulting preconditioners perform well and exhibit good independence on both the mesh size and regularization parameters.

The remainder of this paper is organized as follows. In Section 2, we derive the linear system corresponding to the elliptic optimal control problem and give a brief description of previous work related to this issue. In Section 3, we propose a new preconditioning method and analyze the eigenvalue properties of the preconditioned matrices. In Section 4, numerical results are provided to illustrate the effectiveness of the proposed preconditioning method. Finally, some conclusions are drawn in Section 5.

2. Problem Formulation

The weak formulation of (1) and (2) reads as follows: find $(y,u)\in H_g^1\left(\mathsf{\Omega }\right)\times L_2\left(\mathsf{\Omega }\right)$ with respect to the problem

$$\left\{\begin{array}{c}{\displaystyle \min \frac{1}{2}\left|\right|y-y_d{\left|\right|}_{L_2\left(\mathsf{\Omega }\right)}^2+\frac{\beta }{2}{\left|\right|u\left|\right|}_{L_2\left(\mathsf{\Omega }\right)}^2,}\hfill \\ {\displaystyle \mathrm{s}.\mathrm{t}.{\int }_{\mathsf{\Omega }}\nabla y·\nabla zdx-{\int }_{\mathsf{\Omega }}uzds=0,\forall z\in H_0^1\left(\mathsf{\Omega }\right),}\hfill \end{array}\right.$$

(3)

where $H_g^1\left(\mathsf{\Omega }\right)=\{z:z\in H^1\left(\mathsf{\Omega }\right),z{|}_{\partial \mathsf{\Omega }}=g\}$.

2.1. Discretization

Let $V_0^h\subset H_0^1\left(\mathsf{\Omega }\right)$ be an n dimensional test space with basis $\{{\varphi }_1,\dots ,{\varphi }_n\}$. For the boundary, the basis is extended by defining functions $\{{\varphi }_{n+1},\dots ,{\varphi }_{n+\partial n}\}$. Moreover, $V_g^h=\mathrm{span}\{{\varphi }_1,\dots ,{\varphi }_{n+\partial n}\}\subset H^1\left(\mathsf{\Omega }\right)$. Then any $y_h\in V_g^h\subset H_g^1\left(\mathsf{\Omega }\right)$ can be uniquely determined by $\mathbf{y}={(Y_1,\dots ,Y_n)}^T$ in

$$y_h=\sum _{j=1}^n{Y}_j{\varphi }_j+\sum _{j=n+1}^{n+\partial n}{Y}_j{\varphi }_j,$$

(4)

where $\{Y_{n+1},\dots ,Y_{n+\partial n}\}$ corresponds to the boundary condition $y=g$. Using the same basis, u can be discretized as

$$u_h=\sum _{j=1}^n{U}_j{\varphi }_j,$$

(5)

with the coefficient vector $\mathbf{u}={(U_1,\dots ,U_n)}^T$. It is noted that $u_h=0$ on $\partial \mathsf{\Omega }$.

According to the Galerkin finite element method, problem (3) can be discretized as follows: find $(y_h,u_h)\in V_g^h\times V_0^h$ with respect to the problem

$$\left\{\begin{array}{c}{\displaystyle \min \frac{1}{2}\left|\right|y_h-y_d{\left|\right|}_{L_2\left(\mathsf{\Omega }\right)}^2+\frac{\beta }{2}\left|\right|u_h{\left|\right|}_{L_2\left(\mathsf{\Omega }\right)}^2,}\hfill \\ {\displaystyle \mathrm{s}.\mathrm{t}.{\int }_{\mathsf{\Omega }}\nabla y_h·\nabla z_{h}dx-{\int }_{\mathsf{\Omega }}{u}_h{z}_{h}ds=0,\forall z_h\in V_0^h\left(\mathsf{\Omega }\right).}\hfill \end{array}\right.$$

(6)

Substituting (4) and (5) into (6) results in the matrix form [6,21]

$$\left\{\begin{array}{c}{\displaystyle \underset{\mathbf{y},\mathbf{u}}{\min }\frac{1}{2}{\mathbf{y}}^{T}M\mathbf{y}-{\mathbf{y}}^T\mathbf{b}+\frac{\beta }{2}{\mathbf{u}}^{T}M\mathbf{u},}\hfill \\ {\displaystyle \mathrm{s}.\mathrm{t}.K\mathbf{y}-M\mathbf{u}=\mathbf{d},}\hfill \end{array}\right.$$

(7)

where the elements of the mass matrix M and the stiffness matrix K are

$$M_{ij}={\int }_{\mathsf{\Omega }}{\varphi }_i{\varphi }_{j}d\mathsf{\Omega },K_{ij}={\int }_{\mathsf{\Omega }}\nabla {\varphi }_i·\nabla {\varphi }_{j}d\mathsf{\Omega },i,j=1,\dots ,n,$$

and the elements of the vectors b and d are

$$b_j={\int }_{\mathsf{\Omega }}{y}_d{\varphi }_{j}d\mathsf{\Omega },d_j=-\sum _{i=n+1}^{n+\partial n}{Y}_i{\int }_{\mathsf{\Omega }}\nabla {\varphi }_i·\nabla {\varphi }_{j}d\mathsf{\Omega },j=1,\dots ,n.$$

The Lagrange functional of (7) takes the form

$$L(\mathbf{y},\mathbf{u},\mathbf{p})=\frac{1}{2}{\mathbf{y}}^{T}M\mathbf{y}-{\mathbf{y}}^T\mathbf{b}+\frac{\beta }{2}{\mathbf{u}}^{T}M\mathbf{u}+{\mathbf{p}}^T(K\mathbf{y}-M\mathbf{u}-\mathbf{d}),$$

where $\mathbf{p}={(P_1,\dots ,P_n)}^T$ and $p_h={\sum }_{j=1}^n{P}_j{\varphi }_j$ is the finite element approximation of the Lagrange multiplier p. As a result of first-order optimality, the numerical solution of the elliptical optimal control problem gives the linear system

$$\left(\begin{array}{ccc}M& 0& K\\ 0& \beta M& -M\\ K& -M& 0\end{array}\right)\left(\begin{array}{c}\mathbf{y}\\ \mathbf{u}\\ \mathbf{p}\end{array}\right)=\left(\begin{array}{c}\mathbf{b}\\ \mathbf{0}\\ \mathbf{d}\end{array}\right).$$

(8)

This paper will focus on constructing new preconditioners for this problem.

2.2. Previous Work

In recent years, several preconditioners have been built for (8), such as

$${\mathcal{P}}_{D,i}=\left(\begin{array}{ccc}M& 0& 0\\ 0& \beta M& 0\\ 0& 0& S_i\end{array}\right),i=1,2,$$

where $S_i$ is an approximation of the Schur complement

$$S_0=K{M}^{-1}K+\frac{1}{\beta }M.$$

In [6], Rees et al. approximated it by dropping the second term to make

$$S_1=K{M}^{-1}K.$$

The eigenvalues of $S_1^{-1}{S}_0$ are bounded as $\lambda \left(S_1^{-1}{S}_0\right)\in [\frac{1}{\beta }c{h}^4+1,\frac{1}{\beta }d+1]$ for some constants c and d independent of $\beta$ and mesh size h. The eigenvalue bound and hence the performance of this method was shown to be not independent of the parameters h and $\beta$.

In [7], Pearson and Wathen gave an approximation with the form

$$S_2=(K+\frac{1}{\sqrt{\beta }}M)M^{-1}(K+\frac{1}{\sqrt{\beta }}M).$$

This is the so-called matching strategy. It follows that the eigenvalues of the preconditioned Schur complement satisfy $\lambda \left(S_2^{-1}{S}_0\right)\in [\frac{1}{2},1]$, independently of the values of parameters $\beta$ and h.

3. New Preconditioning Method

3.1. New Preconditioners

The design of the new preconditioner will take advantage of the block structure given in (8). First, we transform the original linear system into its equivalent form

$$\underset{\mathcal{A}}{\underbrace{\left(\begin{array}{ccc}K& -M& 0\\ 0& \beta M& -M\\ M& 0& K\end{array}\right)}}\left(\begin{array}{c}\mathbf{y}\\ \mathbf{u}\\ \mathbf{p}\end{array}\right)=\left(\begin{array}{c}\mathbf{d}\\ \mathbf{0}\\ \mathbf{b}\end{array}\right).$$

(9)

The Schur complement with respect to the (2, 2) block K takes the form

$$S=K+\frac{1}{\beta }M{K}^{-1}M.$$

(10)

It is noted that

$$S=(K+\frac{1}{\sqrt{\beta }}M)K^{-1}(K+\frac{1}{\sqrt{\beta }}M)-\frac{2}{\sqrt{\beta }}M.$$

Inspired by the matching strategy, we approximate the Schur complement as

$$\widehat{S}=(K+\frac{1}{\sqrt{\beta }}M)K^{-1}(K+\frac{1}{\sqrt{\beta }}M).$$

(11)

We note that $S_1$ and $S_2$ are approximations of the Schur complement $S_0$ for the original linear system (8). By contrast, $\widehat{S}$ is an approximation of the Schur complement S for the permuted linear system (9). This is called a matching-strategy-inspired method in a sense that the nature of both terms of the exact Schur complement is captured in the approximation. From the aspect of the decomposition structure, $\widehat{S}$ and $S_2$ differ in the middle part.

Based on the preconditioning theory for block matrices [22], we construct a block triangular preconditioner for $\mathcal{A}$ as the following

$${\mathcal{P}}_T=\left(\begin{array}{ccc}K& -M& 0\\ 0& \beta M& -M\\ 0& 0& \widehat{S}\end{array}\right).$$

(12)

In this paper, the GMRES method will be used combined with the preconditioner ${\mathcal{P}}_T$ for solving (9) as the preconditioned matrix ${\mathcal{P}}_T^{-1}\mathcal{A}$ is unsymmetric.

3.2. Spectral Analysis

We now analyze the spectrum of the preconditioned matrix ${\mathcal{P}}_T^{-1}\mathcal{A}$, which is the same as that of $\mathcal{A}{\mathcal{P}}_T^{-1}$. Note that

$$\mathcal{A}{\mathcal{P}}_T^{-1}=\left(\begin{array}{cc}I& 0\\ *& S{\widehat{S}}^{-1}\end{array}\right),$$

thus $\mu =1$ is an eigenvalue with multiplicity $2n$ and with eigenvectors $e_j\in {\mathbb{R}}^{3n},j=1,\dots ,2n$. The other eigenvalues are the same as that of $S{\widehat{S}}^{-1}$.

Consider the eigenvalue problem

$$S\mathbf{x}=\mu \widehat{S}\mathbf{x},\mathbf{x}\ne 0.$$

(13)

It follows that

$$(K+\frac{1}{\beta }M{K}^{-1}M)\mathbf{x}=\mu (K+\frac{1}{\sqrt{\beta }}M)K^{-1}(K+\frac{1}{\sqrt{\beta }}M)\mathbf{x}.$$

Multiplying by $\beta K^{-1}$ from the left, we have

$$[\beta I+{\left(K^{-1}M\right)}^2]\mathbf{x}=\mu {(\sqrt{\beta }I+K^{-1}M)}^2\mathbf{x}.$$

Then

$${(K^{-1}M+\sqrt{\beta }I)}^{-2}[{\left(K^{-1}M\right)}^2+\beta I]\mathbf{x}=\mu \mathbf{x}.$$

(14)

This implies that for each eigenvalue $\chi$ of $K^{-1}M$, $\frac{{\chi }^2+\beta }{{(\chi +\sqrt{\beta })}^2}$ is an eigenvalue $S{\widehat{S}}^{-1}$. Since $\frac{1}{2}\le \frac{{\chi }^2+\beta }{{(\chi +\sqrt{\beta })}^2}\le 1$ holds for all the $\chi$, $S{\widehat{S}}^{-1}$ possesses the eigenvalue bound

$$\lambda \left(S{\widehat{S}}^{-1}\right)\in [\frac{1}{2},1].$$

(15)

The above results on eigenvalue properties of the preconditioned matrix ${\mathcal{P}}_T^{-1}\mathcal{A}$ are summarized in Proposition 1.

Proposition 1. 

If the block triangular matrix ${\mathcal{P}}_T$ in (12) is taken as a preconditioner for the matrix $\mathcal{A}$ in (9), then 1 is an eigenvalue of the preconditioned matrix ${\mathcal{P}}_T^{-1}\mathcal{A}$ with $2n$ multiplicity and the other eigenvalues are bounded by $[\frac{1}{2},1]$.

According to Proposition 1, the spectrum bound of the preconditioned matrix shows independence on the mesh size and regularization parameter. Notice that the preconditioned system is non-symmetric and the GMRES method is used. Unlike the MINRES method, the spectral analysis is not sufficient to determine the convergence of the GMRES method [23,24]. The eigenvalue bound provided here can help us to gain a better insight into the property of the proposed preconditioner.

4. Numerical Results

In this section, we illustrate the efficiency of the proposed preconditioning method by solving the linear system arising from the elliptic optimal control problem. All the tests were performed using MATLAB R2016b on a machine with Intel Core i5-6200U at 2.30 GHz CPUs and 8 GB of RAM. In all examples, the domain was $\mathsf{\Omega }={(0,1)}^2$ and $g=0$. The regularization parameter was taken to be $\beta ={10}^{-2},{10}^{-4},{10}^{-6}$. $\mathbf{P}\mathbf{1}$ basis functions on a quasi-uniform triangulation of $\mathsf{\Omega }$ were used for state and control variables. Both the GMRES method and MINRES method were terminated when the relative residual in the 2-norm reached a desired tolerance ${10}^{-6}$. In the results, the iteration numbers were exhibited followed by the CPU time (in seconds) in the brackets. A dashed line implies that the corresponding method needs more than 5000 iteration steps.

In order to obtain numerical solutions of the elliptic optimal control problem, we solved the permuted linear system (9) via the GMRES method with the proposed preconditioner ${\mathcal{P}}_T$ in (12), which was denoted as GMRES (${\mathcal{P}}_T$). As comparison, we used the MINRES method combined with the preconditioner ${\mathcal{P}}_{D,i}$ to solve the linear (8). Accordingly, these two methods were denoted MINRES (${\mathcal{P}}_{D,1}$) and MINRES (${\mathcal{P}}_{D,2}$). As a reference, the MINRES method without using a preconditioner was also conducted and it was denoted MINRES (I).

4.1. Test Problem 1

Consider the desired state [7]

$$y_d=\left\{\begin{array}{c}\begin{array}{cc}1,& x\le \frac{1}{2},y\le \frac{1}{2},\\ 0,& \mathrm{elsewhere}.\end{array}\end{array}\right.$$

An illustration of the computed state and control corresponding to DoF = 3267 and $\beta ={10}^{-6}$ is shown in Figure 1.

It can be seen from

Table 1. Comparison of iteration numbers and CPU time for solving test problem 1.

Method	MINRES (I)			MINRES (${\mathcal{P}}_{D,1}$) [6]			MINRES (${\mathcal{P}}_{D,2}$) [7]			GMRES (${\mathcal{P}}_{\mathit{T}}$)
DoF/$\beta$	${10}^{-2}$	${10}^{-4}$	${10}^{-6}$	${10}^{-2}$	${10}^{-4}$	${10}^{-6}$	${10}^{-2}$	${10}^{-4}$	${10}^{-6}$	${\mathbf{10}}^{\mathbf{-}\mathbf{2}}$	${\mathbf{10}}^{\mathbf{-}\mathbf{4}}$	${\mathbf{10}}^{\mathbf{-}\mathbf{6}}$
3267	2278 (1.90)	4662 (3.04)	--	11 (0.29)	27 (0.59)	129 (1.76)	19 (0.42)	25 (0.46)	25 (0.31)	6 (0.28)	8 (0.12)	8 (0.15)
12,675	--	--	--	11 (0.92)	27 (1.66)	132 (7.58)	19 (1.57)	27 (1.82)	27 (1.96)	6 (0.60)	8 (0.73)	8 (0.60)
49,923	--	--	--	13 (4.16)	27 (7.78)	139 (38.4)	21 (7.98)	27 (8.67)	29 (9.18)	6 (2.48)	8 (3.35)	8 (2.63)
198,147	--	--	--	13 (20.2)	27 (34.6)	143 (176)	21 (30.5)	29 (41.5)	29 (41.3)	6 (10.6)	8 (12.9)	8 (13.0)

that the GMRES (${\mathcal{P}}_T$) method requires many fewer iterations and much less CPU time than the MINRES (${\mathcal{P}}_{D,1}$) and MINRES (${\mathcal{P}}_{D,2}$) method. The mesh size independence is exhibited in all cases. Moreover, the GMRES (${\mathcal{P}}_T$) and MINRES (${\mathcal{P}}_{D,2}$) method show independent convergence on the regularization parameter $\beta$, while this is not the case for the MINRES (${\mathcal{P}}_{D,1}$) method. As a reference, it is shown that the required iterations and time of the MINRES method without using a preconditioner are much more than the other methods combined with preconditioners. This indicates the importance of preconditioning for large linear systems.

4.2. Test Problem 2

Consider the desired state [6]

$$y_d=\left\{\begin{array}{c}\begin{array}{cc}{(2x-1)}^2{(2y-1)}^2,& x\le \frac{1}{2},y\le \frac{1}{2},\\ 0,& \mathrm{elsewhere}.\end{array}\end{array}\right.$$

An illustration of the computed state and control is shown in Figure 2.

As shown in

Table 2. Comparison of iteration numbers and CPU time for solving test problem 2.

Method	MINRES (I)			MINRES (${\mathcal{P}}_{D,1}$) [6]			MINRES (${\mathcal{P}}_{D,2}$) [7]			GMRES (${\mathcal{P}}_{\mathit{T}}$)
DoF/$\beta$	${10}^{-2}$	${10}^{-4}$	${10}^{-6}$	${10}^{-2}$	${10}^{-4}$	${10}^{-6}$	${10}^{-2}$	${10}^{-4}$	${10}^{-6}$	${\mathbf{10}}^{\mathbf{-}\mathbf{2}}$	${\mathbf{10}}^{\mathbf{-}\mathbf{4}}$	${\mathbf{10}}^{\mathbf{-}\mathbf{6}}$
3267	1371 (1.12)	1422 (1.13)	1423 (0.84)	45 (0.59)	283 (2.63)	636 (5.64)	11 (0.20)	11 (0.15)	9 (0.14)	8 (0.15)	9 (0.10)	7 (0.09)
12,675	4914 (9.39)	--	--	45 (2.68)	325 (18.8)	1845 (106)	11 (0.62)	11 (0.74)	11 (0.71)	8 (0.56)	9 (0.54)	9 (0.73)
49,923	--	--	--	45 (11.5)	336 (83.9)	2750 (676)	11 (3.07)	11 (3.04)	13 (3.33)	9 (2.55)	9 (2.73)	9 (2.83)
198,147	--	--	--	43 (56.8)	327 (424)	3011 (3923)	11 (15.7)	11 (15.8)	13 (18.6)	9 (13.4)	9 (14.7)	9 (14.5)

, the preconditioner performance here is similar to that presented in test problem 1. The GMRES (${\mathcal{P}}_T$) method and the MINRES (${\mathcal{P}}_{D,2}$) method maintain the mesh size and regularization parameter independence. However, the performance of the MINRES (${\mathcal{P}}_{D,1}$) method depends more on the mesh size as the regularization parameter decreases.

5. Conclusions

In this paper, we proposed a new preconditioning method for the linear system arising from the elliptic optimal control problem. The linear system is first permuted into its equivalent form. Then an approximation for the Schur complement is constructed inspired by the matching strategy. The eigenvalue bounds of the preconditioned matrices are shown to have both mesh size and regularization parameter independence. The efficiency of the proposed preconditioning methods is illustrated by comparison with other existing methods.

In our future work, we will embark on preconditioning the linear systems arising from the PDE-constrained optimization problem with different PDE constraints (for example, parabolic problems), boundary conditions and control types.