ROM-Based Inexact Subdivision Methods for PDE-Constrained Multiobjective Optimization

Abstract

The goal in multiobjective optimization is to determine the so-called Pareto set. Our optimization problem is governed by a parameter-dependent semi-linear elliptic partial differential equation (PDE). To solve it, we use a gradient-based set-oriented numerical method. The numerical solution of the PDE by standard discretization methods usually leads to high computational effort. To overcome this difficulty, reduced-order modeling (ROM) is developed utilizing the reduced basis method. These model simplifications cause inexactness in the gradients. For that reason, an additional descent condition is proposed. Applying a modified subdivision algorithm, numerical experiments illustrate the efficiency of our solution approach.

1. Introduction

Multiobjective optimization plays an important role in many applications, e.g., in industry, medicine, or engineering. One of the mentioned examples is the minimization of costs with simultaneous quality optimization in production or the minimization of CO₂ emission in energy generation and simultaneous cost minimization. These problems lead to multiobjective optimization problems (MOPs), where we want to achieve an optimal compromise with respect to all given objectives at the same time. Normally, the different objectives are contradictory such that there exists an infinite number of optimal compromises. The set of these compromises is called the Pareto set. The goal is to approximate the Pareto set in an efficient way, which turns out to be more expensive than solving a single objective optimization problem.

As multiobjective optimization problems are of great importance, there exist several algorithms to solve them. Among the most popular methods are scalarization methods, which transform MOPs into scalar problems. For example, in the weighted sum method [1,2,3,4], convex combinations of the original objectives are optimized. Another popular approach is to use non-deterministic methods like evolutionary algorithms, cf., e.g., [5]. Furthermore, as multiobjective problems are generalizations of scalar problems, some solution methods can be generalized from the scalar to the multiobjective case [6,7,8].

In addition to the classical methods above, there are set-based strategies for the solution of MOPs. Continuation methods [9,10,11] use the fact that the Pareto set is typically (the projection of) a smooth manifold. Subdivision methods [12,13,14,15] use tools from the area of dynamical systems to generate a covering of the Pareto set via hypercubes. However, especially when the objective functions and their gradients are expensive to evaluate, e.g., as an underlying PDE has to be solved for every evaluation, the computational time of these methods can quickly become very large. In the presence of PDE constraints, surrogate models offer a promising tool to reduce the computational effort significantly [16]. Examples are dimensional reduction techniques such as the Reduced Basis (RB) Method [17,18]. In an offline phase, a low-dimensional surrogate model of the PDE is constructed by using, e.g., the greedy algorithm, cf. [17]. In the online phase, only the RB model is used to solve the PDE, which saves a lot of computing time.

In this article, we combine an extension of the set-oriented method presented in [12] based on inexact gradient evaluations of the objective functions with an RB approach and a discrete empirical interpolation method (DEIM) [19,20] for semi-linear elliptic PDEs. In order to deal with the inexactness introduced by the surrogate model, we combine the first-order optimality conditions for multiobjective optimization problems with error estimates for the RB-DEIM method and derive an additional condition for the descent direction [9] to get a tight superset of the Pareto set. This approach allows us to better control the quality of the result by controlling the errors for the objective functions independently. In order to obtain an even tighter superset of the Pareto set, we update these error estimates in our subdivision algorithm after each iteration step.

The article is organized as follows. In Section 2, we recall the basic concepts of multiobjective optimization problems and review results on descent directions with exact and inexact gradients. Furthermore, we develop a set-oriented method to solve these problems, where only inexact gradient information is utilized. In Section 3, the PDE-constrained multiobjective optimization problem and the underlying semi-linear PDE are introduced. Subsequently, we show how reduced-order modeling can be applied efficiently. In Section 4, numerical results concerning both the subdivision and the modified algorithm are presented. Finally, we give a conclusion and discuss possible future work in Section 5.

2. A Set-Oriented Method for Multiobjective Optimization with Inexact Objective Gradients

In this section, we briefly recall the basic concepts of multiobjective optimization. Furthermore, we develop a set-oriented method to solve these problems, where only inexact gradient information is utilized.

2.1. Multiobjective Optimization

Let ${\mu }_{\mathrm{a}},{\mu }_{\mathrm{b}}\in {\mathbb{R}}^m$ with ${\mu }_{\mathrm{a}}\le {\mu }_{\mathrm{b}}$ be arbitrary. We define the convex and compact parameter set ${\mathcal{P}}_{\mathrm{ad}}=[{\mu }_{\mathrm{a}},{\mu }_{\mathrm{b}}]\subset {\mathbb{R}}^m$. Now, the goal is to solve the constrained multiobjective optimization problem

$$\min \widehat{J}\left(\mu \right)\mathrm{subject}\mathrm{to}(\mathrm{s}.\mathrm{t}.)\mu \in {\mathcal{P}}_{\mathrm{ad}}$$

(1)

with a given objective $\widehat{J}=({\widehat{J}}_1,\dots ,{\widehat{J}}_k):{\mathcal{P}}_{\mathrm{ad}}\to {\mathbb{R}}^k$ and $k>1$.

Compared to scalar optimization, we do not have a natural total order of ${\mathbb{R}}^k$ for $k>1$. Therefore, we cannot expect that there is a single point in ${\mathcal{P}}_{\mathrm{ad}}$ that minimizes all objectives ${\widehat{J}}_i$ simultaneously. For this reason, we make use of the following definition.

Definition 1.

(a)

A point $\overline{\mu }\in {\mathcal{P}}_{\mathrm{ad}}$ is called (globally) Pareto optimal, if there is no $\mu \in {\mathcal{P}}_{\mathrm{ad}}$ satisfying $\widehat{J}\left(\mu \right)\lneqq \widehat{J}\left(\overline{\mu }\right)$. In that case we call $\overline{\mu }$ a Pareto point.

(b)

The set of all Pareto points in ${\mathcal{P}}_{\mathrm{ad}}$ is called Pareto set and is denoted by

$$\mathbb{P}=\left\{\mu \in {\mathcal{P}}_{\mathrm{ad}}|\mu isParetooptimal\right\}.$$

(c)

The image $\widehat{J}\left(\mathbb{P}\right)\subset {\mathbb{R}}^k$ of the Pareto set under $\widehat{J}$ is called the Pareto front.

If $\widehat{J}$ is continuously differentiable on an open set containing ${\mathcal{P}}_{\mathrm{ad}}$, then there exists a first-order necessary condition for Pareto optimality. To formulate this condition, we define the convex, closed, and bounded set

$${∆}_k=\left\{\alpha \in {\mathbb{R}}^k|{\alpha }_i\ge 0\mathrm{and}{\displaystyle \sum _{i=1}^k}{\alpha }_i=1\right\}.$$

Further, the row vector

$$\nabla {\widehat{J}}_i\left(\mu \right)={\left(\frac{\partial {\widehat{J}}_i}{\partial {\mu }_j}\left(\mu \right)\right)}_{1\le j\le m}\in {\mathbb{R}}^{1\times m}$$

stands for the gradient of the i-th objective ${\widehat{J}}_i$ with $i\in \{1,\dots ,k\}$.

Definition 2.

If for a given $\overline{\mu }\in {\mathcal{P}}_{\mathrm{ad}}$ there exists an $\overline{\alpha }={\left({\overline{\alpha }}_i\right)}_{1\le i\le k}\in {∆}_k$ with

$$\begin{array}{cc}\hfill {\sum }_{i=1}^k{\overline{\alpha }}_i\nabla {\widehat{J}}_i\left(\overline{\mu }\right)(\mu -\overline{\mu })& ={(\mu -\overline{\mu })}^{\top }D\widehat{J}{\left(\overline{\mu }\right)}^{\top }\overline{\alpha }\ge 0\mathrm{for}\mathrm{all}\mu \in {\mathcal{P}}_{\mathrm{ad}}\hfill \end{array}$$

(2)

then we call $\overline{\mu }$ Pareto critical, where

$$D\widehat{J}\left(\mu \right)=\left(\begin{array}{c}\nabla {\widehat{J}}_1\left(\overline{\mu }\right)\\ ⋮\\ \nabla {\widehat{J}}_k\left(\mu \right)\end{array}\right)\in {\mathbb{R}}^{k\times m}$$

denotes the Jacobi matrix of $\widehat{J}$ at μ. The set of all Pareto critical points is called the Pareto critical set, denoted by ${\mathbb{P}}_c$.

Now, we recall the first-order necessary optimality conditions for (1).

Theorem 1.

Let $\widehat{J}$ be continuously differentiable and $\overline{\mu }\in {\mathcal{P}}_{\mathrm{ad}}$ Pareto-optimal. Then, $\overline{\mu }$ is Pareto-critical, i.e., it holds $\mathbb{P}\subset {\mathbb{P}}_c$. Condition (2) is called the Karush–Kuhn–Tucker (KKT) condition for multiobjective optimization problems.

Proof. 

The claim follows from ([1] Theorem 3.25) and the specific choice of ${\mathcal{P}}_{\mathrm{ad}}$. □

Remark 1.

(a)

Let $\overline{\mu }$ belong to the interior of ${\mathcal{P}}_{\mathrm{ad}}$, i.e., $\overline{\mu }\in \mathrm{int}\left({\mathcal{P}}_{\mathrm{ad}}\right)=({\mu }_{\mathrm{a}},{\mu }_{\mathrm{b}})$. Then (2) is equivalent to

$${\displaystyle \sum _{i=1}^k{\overline{\alpha }}_i\nabla {\widehat{J}}_i\left(\overline{\mu }\right)={\alpha }^{\top }D\widehat{J}\left(\mu \right)=0in{\mathbb{R}}^{1\times m}}$$

(3)

or

$$D\widehat{J}{\left(\mu \right)}^{\top }\alpha =0in{\mathbb{R}}^m={\mathbb{R}}^{m\times 1};$$

see also in [21].

(b)

Throughout the paper, we only calculate the Pareto critical points in the interior of ${\mathcal{P}}_{\mathrm{ad}}$ and make use of (3). The idea is to choose ${\mathcal{P}}_{\mathrm{ad}}$ sufficiently large so that we get ${\mathbb{P}}_c\subset int\left({\mathcal{P}}_{\mathrm{ad}}\right)$.

(c)

Due to Theorem 1, we have

$$\mathbb{P}\subset {\mathbb{P}}_c=\left\{\mu \in {\mathcal{P}}_{\mathrm{ad}}|\exists \alpha =\alpha \left(\mu \right)\in {∆}_k:D\widehat{J}{\left(\overline{\mu }\right)}^{\top }\alpha =0in{\mathbb{R}}^m\right\}\subset int\left({\mathcal{P}}_{\mathrm{ad}}\right).$$

provided ${\mathbb{P}}_c\subset \mathrm{int}\left({\mathcal{P}}_{\mathrm{ad}}\right)$ holds true. ◊

2.2. Descent Direction with Exact Gradients

Next, we introduce the notion of a descent direction for the vector valued objective function $\widehat{J}$ at a non-Pareto critical point $\mu \notin {\mathbb{P}}_c$. From now on, we assume that $\widehat{J}:{\mathcal{P}}_{\mathrm{ad}}\to {\mathbb{R}}^k$ is continuously differentiable (on an open set containing ${\mathcal{P}}_{\mathrm{ad}}$).

Definition 3.

The vector $v\in {\mathbb{R}}^m$ is a descent direction for $\widehat{J}$ in $\mu \in {\mathcal{P}}_{\mathrm{ad}}$, if we have

$$\nabla {\widehat{J}}_i\left(\mu \right)v\le 0foralli\in \{1,\dots ,k\}$$

and if there is at least one $i\in \{1,\dots ,k\}$ with $\nabla {\widehat{J}}_i\left(\mu \right)v<0$.

One way to compute a descent direction is to solve a constrained quadratic optimization problem in ${\mathbb{R}}^k$ as shown in the following theorem. For a proof we refer to the work in [8]. A similar result was shown in [22].

Theorem 2.

For given $\mu \in \mathrm{int}\left({\mathcal{P}}_{\mathrm{ad}}\right)$ let $\overline{\alpha }=\overline{\alpha }\left(\mu \right)\in {∆}_k$ be a (global) solution of the convex constrained quadratic minimization problem

$$min\frac{1}{2}{\parallel D\widehat{J}{\left(\mu \right)}^{\top }\alpha \parallel }_2^{2}s.t.\alpha \in {∆}_k.$$

(4)

Then, we have either $D\widehat{J}{\left(\mu \right)}^{\top }\overline{\alpha }=0$ or $v=-D\widehat{J}{\left(\mu \right)}^{\top }\overline{\alpha }\in {\mathbb{R}}^m$ is a descent direction for $\widehat{J}$ in μ.

Combined with a backtracking Armijo line search, the descent direction from Theorem 2 can be used to construct the steepest descent method in Algorithm 1.

Algorithm 1: Steepest descent method.

Remark 2.

(a)

If Algorithm 1 terminates after a finite number l of iterations, then ${\mu }^l$ is a Pareto critical point.

(b)

Assume that Algorithm 1 does not stop after a finite number of iterations. Then, every accumulation point $\overline{\mu }$ of the sequence ${\left\{{\mu }^l\right\}}_{l\in \mathbb{N}}$ generated by Algorithm 1 is a Pareto critical point. A proof based on ([7], Theorem 1) can be found in ([23] Theorem 5.2.5). ◊

2.3. Descent Direction with Inexact Gradients

Suppose that we have continuously differentiable approximations ${\widehat{J}}^{\ell }={\left({\widehat{J}}_i^{\ell }\right)}_{1\le i\le k}$ of the objective function $\widehat{J}$ satisfying

$$\underset{\mu \in {\mathcal{P}}_{\mathrm{ad}}}{max}{\parallel \nabla {\widehat{J}}_i\left(\mu \right)-\nabla {\widehat{J}}_i^{\ell }\left(\mu \right)\parallel }_2\le {\epsilon }_i\mathrm{for}i\in \{1,...,k\},$$

(5)

for given tolerances ${\epsilon }_i\ge 0$. By $\mathbb{P}\subset {\mathcal{P}}_{\mathrm{ad}}$ we denote the Pareto set for $\widehat{J}$ and by ${\mathbb{P}}^{\ell }\subset {\mathcal{P}}_{\mathrm{ad}}$ the Pareto set for ${\widehat{J}}^{\ell }$. If we write ${\mathbb{P}}_c$, we mean the Pareto-critical set for $\widehat{J}$ and ${\mathbb{P}}_c^{\ell }$ is the Pareto-critical set for ${\widehat{J}}^{\ell }$. Note that, in general, we have neither ${\mathbb{P}}^{\ell }\subset \mathbb{P}$ nor $\mathbb{P}\subset {\mathbb{P}}^{\ell }$.

In this section, our goal is to compute an approximation of $\mathbb{P}$ based on the approximation ${\widehat{J}}^{\ell }$ of the objective function and the error bounds ${\epsilon }_i$. We begin by investigating the relationship between the KKT conditions of the original objective function and its approximation.

Lemma 1.

Let (5) be satisfied and $\overline{\mu }\in \mathrm{int}\left({\mathcal{P}}_{\mathrm{ad}}\right)$ be Pareto-critical for $\widehat{J}$ with the KKT-condition vector $\overline{\alpha }\in {∆}_k$. Then, it holds that

$${\parallel D{\widehat{J}}^{\ell }{\left(\overline{\mu }\right)}^{\top }\overline{\alpha }\parallel }_2\le \sum _{i=1}^k{\overline{\alpha }}_i{\epsilon }_i={⟨\overline{\alpha },\epsilon ⟩}_2\le {\parallel \epsilon \parallel }_{\infty }$$

(6)

with $\epsilon ={\left({\epsilon }_i\right)}_{1\le i\le k}$, where we set ${⟨\overline{\alpha },\epsilon ⟩}_2={\overline{\alpha }}^{\top }\epsilon$.

Proof. 

From $\overline{\mu }\in \mathrm{int}\left({\mathcal{P}}_{\mathrm{ad}}\right)$ and Remark 1-a) we infer that $D\widehat{J}{\left(\overline{\mu }\right)}^{\top }\overline{\alpha }=0$ holds. Therefore,

$$\begin{array}{cc}\hfill {\parallel D{\widehat{J}}^{\ell }{\left(\overline{\mu }\right)}^{\top }\overline{\alpha }\parallel }_2& ={\parallel D{\widehat{J}}^{\ell }{\left(\overline{\mu }\right)}^{\top }\overline{\alpha }-D\widehat{J}{\left(\overline{\mu }\right)}^{\top }\overline{\alpha }\parallel }_2=\parallel \sum _{j=1}^k(\nabla {\widehat{J}}_j^{\ell }\left(\overline{\mu }\right)-\nabla {\widehat{J}}_j\left(\overline{\mu }\right)){\overline{\alpha }}_j{\parallel }_2\hfill \\ \hfill & \le \sum _{j=1}^k\parallel \nabla {\widehat{J}}_j^{\ell }\left(\overline{\mu }\right)-\nabla {\widehat{J}}_j\left(\overline{\mu }\right){\parallel }_2{\overline{\alpha }}_j\le \sum _{j=1}^k{\epsilon }_j{\overline{\alpha }}_j={⟨\overline{\alpha },\epsilon ⟩}_2\le {\parallel \epsilon \parallel }_{\infty }\hfill \end{array}$$

which gives the desired results. □

Based on estimate (6), we define two approximation sets for the Pareto-critical set ${\mathbb{P}}_c$ of $\widehat{J}$.

Definition 4.

Let us introduce the two sets

$${\mathbb{P}}_1^{\ell }=\left\{\mu \in \mathrm{int}\left({\mathcal{P}}_{\mathrm{ad}}\right)|\underset{\alpha \in {∆}_k}{min}{\parallel D{\widehat{J}}^{\ell }{\left(\mu \right)}^{\top }\alpha \parallel }_2^2\le {\parallel \epsilon \parallel }_{\infty }^2\right\}\subset {\mathcal{P}}_{\mathrm{ad}}$$

and

$${\mathbb{P}}_2^{\ell }=\left\{\mu \in \mathrm{int}\left({\mathcal{P}}_{\mathrm{ad}}\right)|\underset{\alpha \in {∆}_k}{min}\left({\parallel D{\widehat{J}}^{\ell }{\left(\mu \right)}^{\top }\alpha \parallel }_2^2-{⟨\alpha ,\epsilon ⟩}_2^2\right)\le 0\right\}\subset {\mathcal{P}}_{\mathrm{ad}}.$$

Lemma 2.

It holds that

$$\begin{array}{c}\hfill {\mathbb{P}}_c\subset {\mathbb{P}}_2^{\ell },{\mathbb{P}}_c^{\ell }\subset {\mathbb{P}}_2^{\ell }and{\mathbb{P}}_2^{\ell }\subset {\mathbb{P}}_1^{\ell }.\end{array}$$

Proof. 

Let $\overline{\mu }\in {\mathbb{P}}_c$ be a Pareto-critical point of $\widehat{J}$, then there exists $\overline{\alpha }\in {∆}_k$ with $D\widehat{J}{\left(\overline{\mu }\right)}^{\top }\overline{\alpha }=0$. From Lemma 1, it follows that ${\mathbb{P}}_c\subset {\mathbb{P}}_2^{\ell }$. Next, we assume that $\overline{\mu }\in {\mathbb{P}}_c^{\ell }$ is a Pareto-critical point of ${\widehat{J}}^{\ell }$, then there exists $\overline{\alpha }\in {∆}_k$ with $D{\widehat{J}}^{\ell }{\left(\overline{\mu }\right)}^{\top }\overline{\alpha }=0.$ This implies

$$\begin{array}{c}\hfill \underset{\alpha \in {∆}_k}{min}\left(\parallel D{\widehat{J}}^{\ell }{\left(\overline{\mu }\right)}^{\top }\alpha {\parallel }_2^2-{⟨\alpha ,\epsilon ⟩}^2\right)\le \parallel D{\widehat{J}}^{\ell }{\left(\overline{\mu }\right)}^{\top }\overline{\alpha }{\parallel }_2^2-{⟨\overline{\alpha },\epsilon ⟩}_2^2=-{⟨\overline{\alpha },\epsilon ⟩}^2\le 0.\end{array}$$

Therefore, we have $\overline{\mu }\in {\mathbb{P}}_2^{\ell }.$ Let $\overline{\mu }\in {\mathbb{P}}_2^{\ell }$. Then, there exists $\overline{\alpha }\in {∆}_k$ with

$$\begin{array}{c}\hfill \parallel D{\widehat{J}}^{\ell }{\left(\overline{\mu }\right)}^{\top }\overline{\alpha }{\parallel }_2^2-{⟨\overline{\alpha },\epsilon ⟩}_2^2\le 0.\end{array}$$

Thus, we get $\parallel D{\widehat{J}}^{\ell }{\left(\overline{\mu }\right)}^{\top }\overline{\alpha }{\parallel }_2^2\le {⟨\overline{\alpha },\epsilon ⟩}_2^2\le {\parallel \epsilon \parallel }_{\infty }^2$ which implies ${\mathbb{P}}_2^{\ell }\subset {\mathbb{P}}_1^{\ell }$. □

Our goal is to compute the set ${\mathbb{P}}_2^l$ via a descent method like Algorithm 1. To this end, the following theorem presents a modified version of the descent direction (4), which additionally takes the error bounds ${\epsilon }_i$ into account.

Theorem 3.

Let $\epsilon ={\left({\epsilon }_j\right)}_{1\le j\le k}$ with $\epsilon \ge 0$ and $\mu \in \mathrm{int}\left({\mathcal{P}}_{\mathrm{ad}}\right)$ be given. Assume that ${\alpha }_{\epsilon }$ is a minimizer of the quadratic problem

$$\underset{\alpha \in {∆}_k}{min}\left({\parallel D{\widehat{J}}^{\ell }{\left(\mu \right)}^{\top }\alpha \parallel }_2^2-{⟨\alpha ,\epsilon ⟩}_2^2\right).$$

(7)

Then, we have that $\mu \in {\mathbb{P}}_2^{\ell }$ or $v_{\epsilon }=-D{\widehat{J}}^{\ell }{\left(\mu \right)}^{\top }{\alpha }_{\epsilon }\in {\mathbb{R}}^m$ is a descent direction for ${\widehat{J}}^{\ell }$ in μ.

Proof. 

The Lagrange functions for (7) is given as

$$\begin{array}{c}\hfill L:{\mathbb{R}}^k\times \mathbb{R}\times {\mathbb{R}}^k\to \mathbb{R},(\alpha ,\lambda ,\varrho )\mapsto {\parallel D{\widehat{J}}^{\ell }{\left(\mu \right)}^{\top }\alpha \parallel }_2^2-{⟨\alpha ,ϵ⟩}_2^2+\lambda \left(1-\sum _{j=1}^k{\alpha }_j\right)-\sum _{j=1}^k{\varrho }_j{\alpha }_j.\end{array}$$

As ${\alpha }_{\epsilon }$ is a minimizer of (7), we get Lagrangian multipliers $\lambda \in \mathbb{R}$ and $\varrho \in {\mathbb{R}}_{\ge 0}^k$ with

$$\begin{array}{cc}\hfill 2D{\widehat{J}}^{\ell }\left(\mu \right)D{\widehat{J}}^{\ell }{\left(\mu \right)}^{\top }{\alpha }_{\epsilon }-2\epsilon {⟨\epsilon ,{\alpha }_{\epsilon }⟩}_2+\lambda {(-1,...,-1)}^{\top }-\varrho & =0,\hfill \\ \hfill {\varrho }_i\ge 0\mathrm{and}{\left({\alpha }_{\epsilon }\right)}_i{\varrho }_i& =0.\hfill \end{array}$$

(8)

If we multiply (8) with ${\alpha }_{\epsilon }^{\top }$ from the left, we get

$$\begin{array}{c}\hfill 2{\alpha }_{\epsilon }^{\top }D{\widehat{J}}^{\ell }\left(\mu \right)D{\widehat{J}}^{\ell }{\left(\mu \right)}^{\top }{\alpha }_{\epsilon }-2{⟨\epsilon ,{\alpha }_{\epsilon }⟩}_2^2-\lambda \sum _{i=1}^m\left({\left({\alpha }_{\epsilon }\right)}_i-{\left({\alpha }_{\epsilon }\right)}_i{\varrho }_i\right)=0\end{array}$$

which implies

$$\begin{array}{c}\hfill \lambda =2\left(\parallel D{\widehat{J}}^{\ell }{\left(\mu \right)}^{\top }{\alpha }_{\epsilon }{\parallel }_2^2-{⟨{\alpha }_{\epsilon },\epsilon ⟩}_2^2\right).\end{array}$$

First case:$\lambda \le 0$, then $\mu \in {\mathbb{P}}_2^{\ell }$ holds and we are done.

Second case:$\lambda >0$, then $\mu \notin {\mathbb{P}}_2^{\ell }$ holds. In this case, we show that $v_{\epsilon }=-D{\widehat{J}}^{\ell }{\left(\mu \right)}^{\top }{\alpha }_{\epsilon }$ is a descent direction in $\mu$ for every objective function ${\widehat{J}}_j^{\ell }$ with $j=1,...,k$:

Define

$$\begin{array}{c}\hfill K\left(\mu \right)=\left\{D{\widehat{J}}^{\ell }{\left(\mu \right)}^{\top }\alpha |\alpha \in {\mathbb{R}}^k,{\alpha }_i\ge 0\mathrm{and}\sum _{i=1}^k{\alpha }_i=1\right\}.\end{array}$$

If we can show that $w^{\top }{v}_{\epsilon }<0$ holds for every $w\in K$, we know that $v_{\epsilon }$ is a descent direction for every objective function $J_i^{\ell }$ in $\mu$.

Choose $w\in K\left(\mu \right)$. Then, there exists an ${\alpha }_w\in {∆}_k$ with $w=D{\widehat{J}}^{\ell }{\left(\mu \right)}^{\top }{\alpha }_w$, and using (8) we obtain

$$\begin{array}{cc}\hfill w^{\top }{v}_{\epsilon }& ={\left(D{\widehat{J}}^{\ell }{\left(\mu \right)}^{\top }{\alpha }_w\right)}^{\top }{v}_{\epsilon }=-{\alpha }_w^{\top }D{\widehat{J}}^{\ell }\left(\mu \right)D{\widehat{J}}^{\ell }{\left(\mu \right)}^{\top }{\alpha }_{\epsilon }\hfill \\ \hfill & =-{\alpha }_w^{\top }\left(\epsilon {⟨\epsilon ,{\alpha }_{\epsilon }⟩}_2+\frac{1}{2}\lambda (1,...,1)+\frac{1}{2}\varrho \right)=-\left({⟨{\alpha }_w,\epsilon ⟩}_2{⟨{\alpha }_{\epsilon },\epsilon ⟩}_2+\frac{1}{2}\lambda +\frac{1}{2}{⟨{\alpha }_w,\varrho ⟩}_2\right)\hfill \\ \hfill & \le -\frac{1}{2}\left(\lambda +{⟨{\alpha }_w,\varrho ⟩}_2\right)\le -\frac{\lambda }{2}<0.\hfill \end{array}$$

Therefore, $v_{\epsilon }$ is a descent direction in $\mu$ for every objective function ${\widehat{J}}_j^{\ell }$, $j=1,...,k$. □

The descent direction $v_{\epsilon }$ from the previous theorem will be referred to as the modified descent direction. Based on this direction, we can now construct a descent method for the computation of ${\mathbb{P}}_2^l$, which is shown in Algorithm 2.

Algorithm 2: Descent method with inexact gradients.

Remark 3.

(a)

If Algorithm 2 terminates after l iteration steps, then ${\mu }^l$ is contained in ${\mathbb{P}}_2^{\ell }$.

(b)

Assume that Algorithm 2 does not terminate after a finite number of iteration steps. Then, every accumulation point $\overline{\mu }$ of the sequence ${\left\{{\mu }^l\right\}}_{l\in \mathbb{N}}$ generated by Algorithm 2 is in the set ${\mathbb{P}}_2^{\ell }$. A proof based on ([7] Theorem 1) can be found in ([23] Theorem 5.3.5).

(c)

Note that the tolerance ε is constant for all l throughout Algorithm 2. In Section 2.5, we will adapt ε in each iteration. ◊

2.4. Subdivision Algorithm

As mentioned in the introduction, there exist set-based solution methods for MOPs which globally approximate the Pareto set via sets (instead of a finite number of points). Here, we will consider the subdivision algorithm [12,13,15], which computes an approximation of the Pareto set as a covering of hypercubes (or boxes). The idea is to start with a large box containing the Pareto set which is then iteratively subdivided into smaller boxes, while eliminating boxes that do not contain part of the Pareto set.

There are essentially two versions of the subdivision scheme: one is gradient free and, thus, is particularly useful in the case when the evaluation of gradients is computationally expensive. We refer to the work in [12], where this variant is utilized to numerically realize a reduced-order approach for a PDE-constrained multiobjective optimization problem. The other one is directly based on a dynamical systems approach and utilizes gradient information in a similar way to memetic algorithms, see in [8]. Here, we will generalize the latter to the case of inexact gradients.

For a stepsize $t_l>0$, let us formulate a descent step of the optimization procedure by

$${\mu }^{l+1}=a\left({\mu }^l\right)={\mu }^l+t_l{v}^l\mathrm{or}{\mu }^{l+1}=a_{\epsilon }\left({\mu }^l\right)={\mu }^l+t_l{v}_{\epsilon }^l,$$

where $v^l$ and $v_{\epsilon }^l$ are the descent directions given by Theorems 2 and 3, respectively, with the choice $\mu ={\mu }^l$. Depending on the descent step that we use, we either want to compute the Pareto-critical set ${\mathbb{P}}_c$ or the superset ${\mathbb{P}}_2^{\ell }$ or ${\mathbb{P}}_1^{\ell }$ of ${\mathbb{P}}_c$. As these sets are the sets of fixed points for their respective descent step, we want to find the subset $A_{\mathbb{P}}\subset \mathrm{int}\left({\mathcal{P}}_{\mathrm{ad}}\right)$ satisfying $a\left(A_{\mathbb{P}}\right)=A_{\mathbb{P}}$ or $a_{\epsilon }\left(A_{\mathbb{P}}\right)=A_{\mathbb{P}}$.

To generate the set $A_{\mathbb{P}}$, we will use a subdivision method. This method produces an outer approximation of the set $A_{\mathbb{P}}$ in the form of a nested sequence of sets ${\mathcal{B}}_0,{\mathcal{B}}_1,...\subset \mathfrak{P}\left({\mathcal{P}}_{\mathrm{ad}}\right)$, where $\mathfrak{P}\left({\mathcal{P}}_{\mathrm{ad}}\right)$ denotes the power set of ${\mathcal{P}}_{\mathrm{ad}}$ and each ${\mathcal{B}}_l$ is a subset of ${\mathcal{B}}_{l-1}$ in the sense that

$$\bigcup _{B\in {\mathcal{B}}_l}B\subset \bigcup _{B\in {\mathcal{B}}_{l-1}}B$$

holds and ${\mathcal{B}}_l$ consists of finitely many subsets B covering $A_{\mathbb{P}}$ for all $l\in \mathbb{N}$. For each set ${\mathcal{B}}_l$, we define a diameter through $\mathrm{diam}\left({\mathcal{B}}_l\right)={max}_{B\in {\mathcal{B}}_l}\left(\mathrm{diam}\left(B\right)\right)$. Algorithm 3 shows the classical subdivision method (based on Theorem 2) and our modified descent direction (based on Theorem 3).

Algorithm 3: Subdivision algorithm.

Remark 4.

In order to realize the subdivision algorithm numerically we choose a similar way as described in ([13] Remark 2.4). Instead of working explicitly with the centers and radii of the boxes, these are stored within a binary tree in the subdivision step, whereby the memory requirement is noticeably reduced. The selection step is implemented using a certain number of sample points in each box. These sample points are chosen either on an a priori defined grid or randomly within the boxes. Afterwards, $a_{\left(\epsilon \right)}$ is evaluated in these points. For more details, we refer the reader to ([24] Section 5). ◊

2.5. Modified Subdivision Algorithm for Inexact Gradients

In Algorithm 2, we utilize the same error bounds $\epsilon =\left({\epsilon }_1,...,{\epsilon }_k\right)$ with ${\epsilon }_i\ge 0$, $i=1,...,k$, in each iteration step l. Note that the larger the $\epsilon$, the greater the difference between ${\mathbb{P}}_c$ and ${\mathbb{P}}_2^{\ell }$ or ${\mathbb{P}}_1^{\ell }$. In the algorithm, we produce an outer approximation of the set $A_{\mathbb{P}}$ with a nested sequence of sets ${\left\{{\mathcal{B}}_l\right\}}_{l\in \mathbb{N}}$ by

$${\tilde{\mathcal{B}}}_l=\bigcup _{B\in {\mathcal{B}}_l}B\subseteq {\mathcal{P}}_{\mathrm{ad}}.$$

As it holds that ${\tilde{\mathcal{B}}}_l\subset {\mathcal{P}}_{\mathrm{ad}}$, we have

$$max\left\{{\parallel \nabla {\widehat{J}}_i\left(\mu \right)-\nabla {\widehat{J}}_i^{\ell }\left(\mu \right)\parallel }_2|\mu \in {\tilde{\mathcal{B}}}_l\right\}\le max\left\{{\parallel \nabla {\widehat{J}}_i\left(\mu \right)-\nabla {\widehat{J}}_i^{\ell }\left(\mu \right)\parallel }_2|\mu \in {\mathcal{P}}_{\mathrm{ad}}\right\}\mathrm{for}1\le i\le k.$$

Now we modify Algorithm 3 by utilizing the descent directions introduced in Theorem 3 and update $\epsilon$ after every iteration step l to generate a better approximation of the set $A_{\mathbb{P}}$. For updating $\epsilon$, we use the formula

$${\epsilon }_i^l=sup\left\{{\parallel \nabla {\widehat{J}}_i\left(\mu \right)-\nabla {\widehat{J}}_i^{\ell }\left(\mu \right)\parallel }_2|\mu \in {\tilde{\mathcal{B}}}_l\right\}\mathrm{for}1\le i\le k$$

(9)

and set ${\epsilon }^l={\left({\epsilon }_i^l\right)}_{1\le i\le k}$. Due to the nested choice of the box coverings, we have ${\epsilon }_i^{l+1}\le {\epsilon }_i^l$ for $i=1,...,k$ and $l\in \mathbb{N}$. In iteration step l, we generate the descent direction by computing

$${\alpha }_{\epsilon }^l\in arg\; min\left\{{\parallel D{\widehat{J}}^{\ell }{\left(\mu \right)}^{\top }\alpha \parallel }_2^2-{⟨\alpha ,{\epsilon }^l⟩}_2^2|\alpha \in {∆}_k\right\}.$$

(10)

Then, we set

$$v_{\epsilon }^l=-D{\widehat{J}}^{\ell }{\left(\mu \right)}^{\top }{\alpha }_{\epsilon }^l\mathrm{and}{\mu }^{l+1}=a_{\epsilon }^l\left({\mu }^l\right)={\mu }^l+t_l{v}_{\epsilon }^l.$$

Typically, the set ${\mathbb{P}}_2^{\ell }$ is far smaller then the admissible set ${\mathcal{P}}_{\mathrm{ad}}$. Therefore, we expect that the error bounds in (9) become significantly smaller than the ones from (5). Therefore, we expect that we get better results with the modified function $a_{\epsilon }^l$ instead of $a_{\epsilon }$.

3. Multiobjective Optimization of a Semi-Linear Elliptic PDE

In this section, we introduce a multiobjective parameter optimization problem governed by a semi-linear elliptic PDE. Further, we show how reduced-order modeling can be applied efficiently.

3.1. Problem Formulation

Let $\mathsf{\Omega }\subset {\mathbb{R}}^{\mathfrak{d}}$, $\mathfrak{d}\in \{1,2,3\}$, be a bounded domain with Lipschitz-continuous boundary $\mathsf{\Gamma }=\partial \mathsf{\Omega }$. Then, we consider the problem

$$\underset{(y,\mu )}{min}J(y,\mu )=\left(\begin{array}{c}{J}_1(y,\mu )\\ ⋮\\ J_{k-1}(y,\mu )\\ J_k(y,\mu )\end{array}\right)=\frac{1}{2}\left(\begin{array}{c}{\int }_{\mathsf{\Omega }}{|y-y_1^{\mathsf{d}}|}^2\mathrm{d}x\\ ⋮\\ {\int }_{\mathsf{\Omega }}{|y-y_{k-1}^{\mathsf{d}}|}^2\mathrm{d}x\\ {\displaystyle \sum _{j=1}^m{|{\mu }_j-{\mu }_j^{\mathsf{d}}|}^2}\end{array}\right)$$

(11a)

subject to the elliptic boundary value problem

$$-c∆y+by+d{y}^3=f+\sum _{i=1}^m{\mu }_i{\xi }_i\mathrm{in}\mathsf{\Omega },c\frac{\partial y}{\partial \mathbf{n}}+y=g\mathrm{on}\mathsf{\Gamma },$$

(11b)

where $y\in V=H^1\left(\mathsf{\Omega }\right)$ is the state variable and $\mu \in {\mathcal{P}}_{\mathrm{ad}}=[{\mu }_{\mathrm{a}},{\mu }_{\mathrm{b}}]$ the parameter. We suppose that $g\in L^r\left(\mathsf{\Gamma }\right)$ with $r>\mathfrak{d}-1$, ${\xi }_1,...,{\xi }_m,f\in H=L^2\left(\mathsf{\Omega }\right)$, ${\mu }^{\mathsf{d}}=\left({\mu }_j^{\mathsf{d}}\right)\in {\mathbb{R}}^m$, and $y_1^{\mathsf{d}},...,y_{k-1}^{\mathsf{d}}\in H$. Moreover, b, c, and d are non-negative constants with $c>0$.

As $\mathsf{\Omega }$ is a bounded connected open set with smooth boundary, it is known that V is a Hilbert space endowed with the inner product

$${⟨\phi ,\psi ⟩}_V={\int }_{\mathsf{\Omega }}\nabla \phi ·\nabla \psi \mathrm{d}x+{\int }_{\mathsf{\Gamma }}\phi \psi \mathrm{d}s\mathrm{for}\phi ,\psi \in V$$

and the induced norm ${\parallel \phi \parallel }_V={⟨\phi ,\phi ⟩}_V^{1/2}$ for $\phi \in V$, see ([25], p. 133) for instance.

3.2. The Parameter Dependent Semi-Linear Elliptic PDE

In this subsection, we study the state equation (11b). First, we define the nonlinear operator $\mathcal{A}:V\to V^{\prime }$ by

$${⟨\mathcal{A}\left(y\right),\phi ⟩}_{V^{\prime },V}={\int }_{\mathsf{\Omega }}c\nabla y·\nabla \phi +\left(by+d{y}^3\right)\phi \mathrm{d}x+{\int }_{\mathsf{\Gamma }}y\phi \mathrm{d}s\mathrm{for}y,\phi \in V.$$

Recall that $\mathfrak{d}\in \{1,2,3\}$ implies $V\hookrightarrow L^6\left(\mathsf{\Omega }\right)$, cf. ([26] Section 7). Therefore, the operator $\mathcal{A}$ is well defined. Moreover, for $\mu \in {\mathcal{P}}_{\mathrm{ad}}$ the functional $b_{\mu }\in V^{\prime }$ is given by

$${⟨b_{\mu },\phi ⟩}_{V^{\prime },V}={\int }_{\mathsf{\Omega }}\left(f+\sum _{i=1}^m{\mu }_i{\xi }_i\right)\phi \mathrm{d}x+{\int }_{\mathsf{\Gamma }}g\phi \mathrm{d}s\mathrm{for}\phi \in V.$$

Now, we define a weak solution to the state Equation (11b).

Definition 5.

A weak solution of (11b) is a function $y\in V$ satisfying

$${⟨\mathcal{A}\left(y\right),\phi ⟩}_{V^{\prime },V}={⟨b_{\mu },\phi ⟩}_{V^{\prime },V}forall\phi \in V.$$

(12)

The following result is proved, e.g., in ([26] Section 4.2.3).

Proposition 1.

For a fixed parameter $\mu \in {\mathbb{R}}^m$, there exists a unique solution $y\in V$ to (11b). This solution is even continuous on $\overline{\mathsf{\Omega }}$, and for a constant $c_{\infty }$ it holds that

$${\parallel y\parallel }_V+{\parallel y\parallel }_{C\left(\overline{\mathsf{\Omega }}\right)}\le c_{\infty }\left({\parallel g\parallel }_{L^r\left(\mathsf{\Gamma }\right)}+\parallel f+\sum _{i=1}^m{\mu }_i{\xi }_i{\parallel }_H\right).$$

Remark 5.

We define the state space $\mathcal{Y}=V\cap C\left(\overline{\mathsf{\Omega }}\right)$, which is a Banach space endowed with the natural norm

$${\parallel \phi \parallel }_{\mathcal{Y}}={\parallel \phi \parallel }_V+{\parallel \phi \parallel }_{C\left(\overline{\mathsf{\Omega }}\right)}for\phi \in \mathcal{Y}.$$

Motivated by Proposition 1 , we define the parameter-to-state mapping $\mathcal{S}:{\mathcal{P}}_{\mathrm{ad}}\to \mathcal{Y}$ as follows: For a given parameter $\mu \in {\mathcal{P}}_{\mathrm{ad}}$, the function $y=\mathcal{S}\left(\mu \right)\in \mathcal{Y}$ is the solution to (11b). It follows by standard arguments that $\mathcal{S}$ is continuously Fréchet-differentiable, see ([23] Sections 2 and 4). ◊

3.3. Reduced Formulation and Adjoint Approach

Utilizing the parameter-to-state mapping $\mathcal{S}$, we define the reduced cost functional

$$\widehat{J}\left(\mu \right)=J(\mathcal{S}\left(\mu \right),\mu )=\left(\begin{array}{c}{\widehat{J}}_1\left(\mu \right)\\ ⋮\\ {\widehat{J}}_k\left(\mu \right)\end{array}\right)\mathrm{for}\mu \in {\mathcal{P}}_{\mathrm{ad}}$$

with

$${\widehat{J}}_i\left(\mu \right)=\frac{1}{2}{\int }_{\mathsf{\Omega }}|\left(\mathcal{S}\left(\mu \right)\right)\left(x\right)-y_i^{\mathsf{d}}\left(x\right){|}^2\mathrm{d}x\mathrm{for}1\le i\le k-1$$

and ${\displaystyle {\widehat{J}}_k\left(\mu \right)=\sum _{j=1}^m{|{\mu }_j-{\mu }_j^{\mathsf{d}}|}^2/2}$. Now, the reduced problem is given as

$$min\widehat{J}\left(\mu \right)\mathrm{s}.\mathrm{t}.\mu \in {\mathcal{P}}_{\mathrm{ad}}.$$

(13)

If $\overline{\mu }\in {\mathcal{P}}_{\mathrm{ad}}$ is a locally optimal solution to (13), then the pair $(\mathcal{S}\left(\overline{\mu }\right),\overline{\mu })$ is a locally optimal solution to (11). Conversely, if $(\mathcal{S}\left(\overline{\mu }\right),\overline{\mu })\in \mathcal{Y}\times {\mathcal{P}}_{\mathrm{ad}}$ solves (11) locally, the parameter $\overline{\mu }$ is a locally optimal solution to (13).

To apply the subdivision algorithm, the reduced objective function $\widehat{J}$ has to be Fréchet-differentiable, following immediately from the fact that the parameter-to-state operator $\mathcal{S}$ is Fréchet-differentiable, cf. Remark 5. The gradient of $\widehat{J}$ can be expressed by introducing adjoint variables. For that purpose, we define the operators ${\mathcal{B}}_i:V\times V\to V^{\prime }$, $1\le i\le k-1$, as

$${⟨{\mathcal{B}}_i(p,y),\phi ⟩}_{V^{\prime },V}={\int }_{\mathsf{\Omega }}c\nabla p·\nabla \phi +\left(b+3d{y}^2\right)p\phi \mathrm{d}x+{\int }_{\mathsf{\Gamma }}p\phi \mathrm{d}x-\underset{G}{\int }(y-y_i^{\mathsf{d}})\phi \mathrm{d}x$$

for $p,y,\phi \in V$. Next, we define adjoint variables.

Definition 6.

Let a parameter $\mu \in {\mathcal{P}}_{\mathrm{ad}}$ be given and $y\left(\mu \right)=\mathcal{S}\left(\mu \right)\in \mathcal{Y}$. For every $i\in \{1,\dots ,k-1\}$, we call the solution $p_i\left(\mu \right)\in V$ to

$${⟨{\mathcal{B}}_i(p_i\left(\mu \right),y\left(\mu \right)),\phi ⟩}_{V^{\prime },V}=0forall\phi \in V$$

(14)

the adjoint variable associated with the objective${\widehat{J}}_i$.

Remark 6.

(a)

Notice that (14) is the weak formulation of the elliptic problem

$$\begin{array}{cccc}\hfill -c∆p_i\left(\mu \right)+\left(b+3dy{\left(\mu \right)}^2\right)p_i\left(\mu \right)& =y\left(\mu \right)-y_i^{\mathsf{d}}\hfill & \hfill & \mathrm{in}\mathsf{\Omega },\hfill \\ \hfill c\frac{\partial p_i}{\partial \mathbf{n}}\left(\mu \right)+p_i\left(\mu \right)& =0\hfill & \hfill & \mathrm{on}\mathsf{\Gamma }\hfill \end{array}$$

(15)

for $i=1,\dots ,k-1$.

(b)

Applying the Lax–Milgram lemma (see, e.g., ([26] Section 2)) one can show that (14) has a unique solution $p_i\left(\mu \right)\in V$ for all $\mu \in {\mathcal{P}}_{\mathrm{ad}}$ and $i=1,\dots ,k-1$. ◊

Now, we can express the gradient of the reduced cost functional as follows:

$$\nabla {\widehat{J}}_i\left(\mu \right)=\left(\begin{array}{c}{\int }_{\mathsf{\Omega }}{\xi }_1{p}_i\left(\mu \right)\mathrm{d}x\\ ⋮\\ {\int }_{\mathsf{\Omega }}{\xi }_m{p}_i\left(\mu \right)\mathrm{d}x\end{array}\right)\mathrm{for}1\le i\le k-1,\nabla {\widehat{J}}_k\left(\mu \right)=\mu -{\mu }^{\mathsf{d}},$$

where $y\left(\mu \right)\in \mathcal{Y}$ and $p_i\left(\mu \right)\in V$, $i\in \{1,\dots ,k-1\}$, solve (12) and (14), respectively.

3.4. Finite Element (FE) Galerkin Discretization

Let us briefly introduce a standard finite element (FE) method based on a triangulation of the spatial domain $\mathsf{\Omega }$. Here, we utilize piecewise linear FE ansatz functions ${\phi }_1,\dots ,{\phi }_N\in V$, which are linearly independent. We define the finite-dimensional subspace $V^N=\mathrm{span}\{{\phi }_1,...,{\phi }_N\}\subset V$ supplied with the same topology as in V.

Next, we replace (12) by a FE Galerkin scheme: For each $\mu \in {\mathcal{P}}_{\mathrm{ad}}$, the FE solution $y^N\left(\mu \right)\in V^N$ solves

$${⟨\mathcal{A}\left(y^N\left(\mu \right)\right),{\phi }_i⟩}_{V^{\prime },V}={⟨b_{\mu },{\phi }_i⟩}_{V^{\prime },V}\mathrm{for}i=1,\dots ,N.$$

(16)

It follows by the same arguments as for (12) that the FE problem (16) has a unique solution $y^N=y^N\left(\mu \right)$ for every $\mu \in {\mathcal{P}}_{\mathrm{ad}}$. Therefore, the parameter-to-state FE mapping ${\mathcal{S}}^N:{\mathcal{P}}_{\mathrm{ad}}\to V^N$, $\mu \mapsto {\mathcal{S}}^N\left(\mu \right)=y^N\left(\mu \right)$ is well defined. For $y^N\in V^N$ there exist coefficients ${\mathrm{y}}_i^N$, $i=1,\dots ,N$, satisfying

$$y^N=\sum _{i=1}^N{\mathrm{y}}_i^N{\phi }_i\mathrm{for}\mathit{x}\in \mathsf{\Omega }.$$

(17)

Inserting (17) into (16), we can express (16) as a nonlinear algebraic system. For that purpose, we introduce the $N\times N$-matrices

$$\mathrm{K}=\left(\left({\int }_{\mathsf{\Omega }}\nabla {\phi }_j·\nabla {\phi }_i\mathrm{d}x\right)\right),\mathrm{M}=\left(\left({\int }_{\mathsf{\Omega }}{\phi }_j{\phi }_i\mathrm{d}x\right)\right),\mathrm{Q}=\left(\left({\int }_{\mathsf{\Gamma }}{\phi }_j{\phi }_i\mathrm{d}s\right)\right),$$

the N-vectors

$${\mathrm{y}}^N=\left({\mathrm{y}}_i^N\right),{\mathrm{F}}_{\mu }=\left({\int }_{\mathsf{\Omega }}(f+\sum _{j=1}^m{\mu }_j{\xi }_j\right){\phi }_i\mathrm{d}x),\mathrm{G}=\left({\int }_{\mathsf{\Gamma }}g{\phi }_i\mathrm{d}s\right),$$

and the nonlinearity function $\mathrm{H}:{\mathbb{R}}^N\to {\mathbb{R}}^N$ given as

$$\mathrm{H}\left(\mathrm{v}\right)=\left({\int }_{\mathsf{\Omega }}{\left({\sum }_{j=1}^N{\mathrm{v}}_j{\phi }_j\right)}^3{\phi }_i\mathrm{d}x\right)\mathrm{for}\mathrm{v}=\left({\mathrm{v}}_1,\dots ,{\mathrm{v}}_N\right)\in {\mathbb{R}}^N.$$

Inserting (17) into (16), we end up with the following nonlinear system:

$$\left(c\mathrm{K}+b\mathrm{M}+\mathrm{Q}\right){\mathrm{y}}^N+d\mathrm{H}\left({\mathrm{y}}^N\right)={\mathrm{F}}_{\mu }+\mathrm{G}\in {\mathbb{R}}^N.$$

(18)

Remark 7.

The difficulty of (18) lies in the fact that we cannot assemble $\mathrm{H}\left({\mathrm{y}}^N\right)$ efficiently in terms of a matrix-vector multiplication. Therefore, we use mass lumping to compute $\mathrm{H}\left({\mathrm{y}}^N\right)$ approximately. For an introduction into mass lumping see, e.g., ([27] Section 15) or ([28] Section 17.2). With that we can write (18) as a root finding problem of

$$\left(c\mathrm{K}+b\mathrm{M}+\mathrm{Q}\right){\mathrm{y}}^N+d\tilde{M}{\left({\mathrm{y}}^N\right)}^3={\mathrm{F}}_{\mu }+\mathrm{G}\in {\mathbb{R}}^N$$

(19)

with ${\left({\mathrm{y}}^N\right)}^3={\left({\left({\mathrm{y}}_i^N\right)}^3\right)}_{1\le i\le N}$ and the lumped mass matrix $\tilde{M}$ defined by

$$\begin{array}{c}\hfill {\tilde{M}}_{kj}:={\delta }_{kj}\sum _{l=1}^N{M}_{kl}.\end{array}$$

Further details can be found in [23,29]. Let us refer to in [30], where mass lumping is utilized in optimal control. ◊

Now, the FE objectives are given as

$${\widehat{J}}_i^N\left(\mu \right)=\frac{1}{2}{\int }_{\mathsf{\Omega }}|{\mathcal{S}}^N\left(\mu \right)-y_i^{\mathsf{d}}{|}^2\mathrm{d}x\mathrm{for}i=1,...,k-1,{\widehat{J}}_k^N\left(\mu \right)=\sum _{j=1}^m|{\mu }_j-{\mu }_j^{\mathsf{d}}{|}^2.$$

3.5. Reduced-Order Modelling (ROM)

To generate the Pareto-critical set of the MOP (11), we need to evaluate the reduced objectives ${\widehat{J}}_i$, $i=1,\dots ,k$, and their gradients many times. Therefore, the state and adjoint equations have to be solved numerically very often. Therefore, the use of ROM is a suitable option. In this paper, we will use the Reduced Basis (RB) method. The main idea is to construct a low-dimensional (i.e., $\ell \ll N$) subspace $V^{\ell }$ of the FE space $V^N$ spanned by FE solutions of the state and adjoint equation for appropriately chosen parameters $\mu \in {\mathcal{P}}_{\mathrm{ad}}$. Here, this strategy is realized by greedy algorithms. We refer to the works in [17,18,31] for a general explanation and to ([23] Section 3.2) for our specific problem (11). This is an iterative procedure where in each iteration FE solutions of the state and adjoint equation at a specific parameter value are added to the basis. An essential ingredient of greedy algorithms is the choice of an error indicator ${\eta }_{\ell }\left(\mu \right)$. Here, we use the maximal true error between the FE and the ROM gradients, i.e., we set

$${\eta }_{\ell }\left(\mu \right):=\underset{1\le i\le k}{max}{\parallel \nabla {\widehat{J}}_i^N\left(\mu \right)-\nabla {\widehat{J}}_i^{\ell }\left(\mu \right)\parallel }_2.$$

(20)

Our subdivision scheme is based on gradient information. To be able to generate ${\mathbb{P}}_1^{\ell }$ or ${\mathbb{P}}_2^{\ell }$ for the approximation of $\mathbb{P}$, we have to ensure that the approximated objective function ${\widehat{J}}^{\ell }$ based on the ROM model satisfies the inequality in (5). The idea is to generate a new basis element in every step until the maximum error ${\eta }_{\ell }\left(\mu \right)$ on a discrete training set $S_{train}$, which approximates ${\mathcal{P}}_{\mathrm{ad}}$ good enough, is smaller than a tolerance ${\epsilon }_{tol}$. For more details, see Algorithm 4.

Algorithm 4: Greedy algorithm.

As $V^{\ell }$ is a subset of V, we endow $V^{\ell }$ with the V-topology. Due to ${\psi }_j\in V^N$ ($1\le j\le \ell$), there exists a coefficient matrix $\mathsf{\Psi }\in {\mathbb{R}}^{N\times \ell }$ such that

$${\psi }_j\left(x\right)=\sum _{i=1}^N{\mathsf{\Psi }}_{ij}{\phi }_i\left(x\right)\mathrm{for}x\in \mathsf{\Omega }.$$

(21)

Now, we replace (16) by an RB Galerkin scheme: For each $\mu \in {\mathcal{P}}_{\mathrm{ad}}$, the RB solution $y^{\ell }\left(\mu \right)\in V^{\ell }$ solves

$${⟨\mathcal{A}\left(y^{\ell }\left(\mu \right)\right),{\psi }_i⟩}_{V^{\prime },V}={⟨b_{\mu },{\psi }_i⟩}_{V^{\prime },V}\mathrm{for}i=1,\dots ,\ell .$$

(22)

We suppose that (22) has a unique solution $y^{\ell }=y^{\ell }\left(\mu \right)$ for every $\mu \in {\mathcal{P}}_{\mathrm{ad}}$. Therefore, the parameter-to-state RB mapping ${\mathcal{S}}^{\ell }:{\mathcal{P}}_{\mathrm{ad}}\to V^{\ell }$, $\mu \mapsto {\mathcal{S}}^{\ell }\left(\mu \right)=y^{\ell }\left(\mu \right)$ is well defined.

Inserting (21) and $y^{\ell }\left(\mu \right)={\sum }_{i=1}^N{\mathrm{y}}_i^{\ell }\left(\mu \right){\psi }_i$ into (22), we derive the nonlinear algebraic system (cf. (18))

$$\left(c{\mathrm{K}}^{\ell }+b{\mathrm{M}}^{\ell }+{\mathrm{Q}}^{\ell }\right)+d{\mathsf{\Psi }}^{\top }\mathrm{H}\left(\mathsf{\Psi }{\mathrm{y}}^{\ell }\right)={\mathrm{F}}_{\mu }^{\ell }+{\mathrm{G}}^{\ell }\in {\mathbb{R}}^l(\ell \ll N)$$

(23)

with the $\ell \times \ell$ matrices ${\mathrm{K}}^{\ell }={\mathsf{\Psi }}^{\top }\mathrm{K}\mathsf{\Psi }$, ${\mathrm{M}}^{\ell }={\mathsf{\Psi }}^{\top }\mathrm{M}\mathsf{\Psi }$, and ${\mathrm{Q}}^{\ell }={\mathsf{\Psi }}^{\top }\mathrm{Q}\mathsf{\Psi }$ and the ℓ-vectors ${\mathrm{F}}_{\mu }^{\ell }={\mathsf{\Psi }}^{\top }{\mathrm{F}}_{\mu }$ and ${\mathrm{G}}^{\ell }={\mathsf{\Psi }}^{\top }\mathrm{G}$.

Remark 8.

As mentioned in Remark 7, we apply mass lumping to evaluate the nonlinear function $\mathrm{H}$ more efficiently. With that we can write (23) as a root finding problem of

$$\left(c{\mathrm{K}}^{\ell }+b{\mathrm{M}}^{\ell }+{\mathrm{Q}}^{\ell }\right){\mathrm{y}}^{\ell }+d{\tilde{\mathrm{M}}}^{\ell }{\left(\mathsf{\Psi }{\mathrm{y}}^{\ell }\right)}^3={\mathrm{F}}_{\mu }^{\ell }+{\mathrm{G}}^{\ell }\in {\mathbb{R}}^l(\ell \ll N)$$

(24)

with ${\left(\mathsf{\Psi }{y}^{\ell }\right)}^3={\left(\left({\left(\mathsf{\Psi }{y}^{\ell }\right)}_i^3\right)\right)}_{1\le i\le N}$ and the $\ell \times N$ matrix ${\tilde{\mathrm{M}}}^{\ell }={\mathsf{\Psi }}^{\top }\tilde{\mathrm{M}}$. However, in the RB Galerkin scheme the evaluation of the nonlinearity is still as costly as in the FE case. Here, discrete empirical interpolation (DEIM) is applied, cf. [19,20]. We skip the detailed description here and refer the reader to ([23] Section 3.2). ◊

3.6. Convergence Analysis

We prove the convergence of the RB solution with mass lumping to the weak solution of the state and adjoint equation.

Remark 9.

As the FE space is a finite dimensional space, the error for the state and adjoint equation converges for increasing dimension of the RB space to zero. Therefore, the RB solution converges for increasing accuracy in the Greedy algorithm to the FE solution. We skip a detailed description of the proof here and refer the reader to ([23], Section 3.2). ◊

Theorem 4.

Let ${\left(S_j\right)}_j\subset {\mathcal{P}}_{\mathrm{ad}}$ with $S_j\subseteq S_{j+1}$ be a growing sequence of training sets and choose a monotone sequence ${\left({\epsilon }_j\right)}_j\subset {\mathbb{R}}_{>0}$ satisfying

$$\begin{array}{c}\hfill {\epsilon }_j\to 0\mathrm{for}j\to \infty \mathrm{and}{\epsilon }_j\ge {\epsilon }_{j+1}.\end{array}$$

Then, we get

$$\begin{array}{c}\hfill \underset{j\to \infty }{lim\; sup}\left\{{\parallel y\left(\mu \right)-y^{\ell }\left(\mu \right)\parallel }_V|\mu \in {\mathcal{P}}_{\mathrm{ad}}\right\}=0,\underset{j\to \infty }{lim\; sup}\left\{{\parallel p\left(\mu \right)-p^{\ell }\left(\mu \right)\parallel }_V|\mu \in {\mathcal{P}}_{\mathrm{ad}}\right\}=0.\end{array}$$

Proof. 

Let $\mu \in {\mathcal{P}}_{\mathrm{ad}}$ be an arbitrary parameter. It holds that

$$\begin{array}{c}\hfill {\parallel y\left(\mu \right)-y^{\ell }\left(\mu \right)\parallel }_V\le {\parallel y\left(\mu \right)-y^N\left(\mu \right)\parallel }_V+{\parallel y^N\left(\mu \right)-y^{\ell }\left(\mu \right)\parallel }_V\end{array}$$

with $N>\ell$. From Theorem A1 (see Appendix A), we infer that the first summand converges to zero for increasing N. For the second summand, we refer the reader to ([23] Section 3.2.5); in this section we proved the convergence of the RB solution to the FE solution. For the adjoint equation, we can do the same and the claim follows. □

4. Numerical Experiments

In this section, we use our algorithm to solve multiobjective optimization problems with PDE constraints and interpret the numerical results. All computations were executed on a computer with a 2.9 GHz Intel Core i7 CPU, 8 GB of RAM, and an Intel HD Graphic 4000 1536 MB GPU. The algorithms were implemented in Matlab R2017b. For the subdivision method, we used the implementation from https://math.uni-paderborn.de/en/ag/chair-of-applied-mathematics/research/software.

In this example, we will numerically investigate the application of the modified subdivision algorithm presented in Section 2.5 to the PDE-constrained multiobjective optimization problem using the RB-DEIM solver from Section 3.5. For the underlying PDE, we set $\mathfrak{d}=2$, $\mathsf{\Omega }={(0,1)}^2$ with elements $x=(x_1,x_2)$, ${\mathcal{P}}_{\mathrm{ad}}={[-2,2]}^2$, and $b=c=d=1$; the right-hand side $f\left(x\right)=x_1^2+x_2^2-4+{(x_1^2+x_2^2)}^3$, $m=2$, ${\xi }_1\left(x\right)=-25·{\mathbb{1}}_{x_1>0.5}\left(x\right)$, and ${\xi }_2\left(x\right)=25·{\mathbb{1}}_{x_1\le 0.5}\left(x\right)$; and the boundary condition $g\left(x\right)=2·{\mathbb{1}}_{x_1=1}\left(x\right)+2·{\mathbb{1}}_{x_2=1}\left(x\right)+x_1^2+x_2^2$. This leads to the following PDE:

$$-∆y+y+y^3=f+{\mu }_1{\xi }_1+{\mu }_2{\xi }_2\mathrm{in}\mathsf{\Omega },\frac{\partial y}{\partial \mathit{n}}+y=g\mathrm{on}\partial \mathsf{\Omega }.$$

(25)

In Figure 1, the corresponding solutions of the state equation are shown for three values of $\mu$.

Figure 1. FE solutions of (25) for parameters (a) $\mu =(0,0)$, (b) $\mu =(0,1)$ and (c) $\mu =(1,0)$.

In [23], we have already observed that the error between the FE- and RB-DEIM-solution of the state and adjoint equation decreases if the FE grids get finer. We skip the detailed description here and refer the reader to ([23] Section 5).

Notice that $y\left(\mathit{x}\right)=x_1^2+x_2^2$ solves (25) for $\mu =(0,0)$. For the FE-solver, we used linear finite elements with $∆H_{max}=0.04$ and the finite elements have 762 degrees of freedom. We choose the following two objective functions:

$$\begin{array}{c}\hfill {\widehat{J}}_1\left(\mu \right)=\frac{1}{2}{\int }_{\mathsf{\Omega }}|y\left(\mu \right)-y^{\mathsf{d}}{|}^2\mathrm{d}\mathit{x},{\widehat{J}}_2\left(\mu \right)=\frac{1}{2}\sum _{j=1}^2{|{\mu }_j-{\mu }_j^{\mathsf{d}}|}^2\end{array}$$

with ${\mu }^{\mathsf{d}}=(1,1)$. For the desired state $y^{\mathsf{d}}$, we take the FE solution for $\mu =0$, i.e., $y^{\mathsf{d}}=y^N\left(0\right)$. Thus, $y^{\mathsf{d}}={\sum }_{i=1}^N{\mathrm{y}}_i^{\mathsf{d}}{\phi }_i$ is a piecewise linear approximation of $x_1^2+x_2^2$. The associated FE objectives are now given as

$$\begin{array}{c}\hfill {\widehat{J}}_1^N\left(\mu \right)=\frac{1}{2}{\left({\mathrm{y}}^N\left(\mu \right)-{\mathrm{y}}^{\mathsf{d}}\right)}^{\top }\mathrm{M}\left(y^N\left(\mu \right)-{\mathrm{y}}^{\mathsf{d}}\right),{\widehat{J}}_2^N\left(\mu \right)=\frac{1}{2}\sum _{j=1}^2{|{\mu }_j-{\mu }_j^{\mathsf{d}}|}^2\end{array}$$

with ${\mathrm{y}}^{\mathsf{d}}={\left({\mathrm{y}}_i^{\mathsf{d}}\right)}_{1\le i\le N}$. The gradients are

$$\begin{array}{c}\hfill \nabla {\widehat{J}}_1^N\left(\mu \right)=\left(\begin{array}{c}{\mathrm{p}}^N{\left(\mu \right)}^{\top }{\overline{F}}_1\\ {\mathrm{p}}^N{\left(\mu \right)}^{\top }{\overline{F}}_2\end{array}\right),\nabla {\widehat{J}}_2^N\left(\mu \right)=\mu -{\mu }^{\mathsf{d}}\end{array}$$

where $p^N\left(\mu \right)={\sum }_{j=1}^N{\mathrm{p}}_j^N\left(\mu \right){\phi }_j$ is the FE solution to (15), ${\mathrm{p}}^N\left(\mu \right)={\left({\mathrm{p}}_j^N\left(\mu \right)\right)}_{1\le j\le N}$ and ${\overline{F}}_i={\left({\int }_{\mathsf{\Omega }}{\phi }_j{\xi }_i\mathrm{d}\mathit{x}\right)}_{1\le j\le N}$. The associated RB objective functions have the form

$$\begin{array}{c}\hfill {\widehat{J}}_1^{\ell }\left(\mu \right)=\frac{1}{2}{\left({\mathrm{y}}^{\ell }\left(\mu \right)-{\mathrm{y}}^{\mathsf{d},\ell }\right)}^{\top }{\mathrm{M}}^{\ell }\left({\mathrm{y}}^{\ell }\left(\mu \right)-{\mathrm{y}}^{\mathsf{d},\ell }\right),{\widehat{J}}_2^{\ell }\left(\mu \right)=\frac{1}{2}\sum _{j=1}^m{|{\mu }_j-{\mu }_j^{\mathsf{d}}|}^2\end{array}$$

with $y^{\mathsf{d},\ell }={\mathsf{\Psi }}^{\top }{\mathrm{y}}^{\mathsf{d}}$. In [23], we have already observed good agreement between the approximated Pareto critical sets with the FE- and RB-DEIM-solver, if the error between the gradients is sufficiently small. However, we cannot always guarantee this agreement. Thus, we will instead use the supersets ${\mathbb{P}}_1^{\ell }$ and ${\mathbb{P}}_2^{\ell }$ from Section 2.3 for the approximation of $\mathbb{P}$ (and ${\mathbb{P}}_c$), which is only based on the reduced objective function ${\widehat{J}}^{\ell }$ and the error bounds $\epsilon$. To generate ${\mathbb{P}}_1^{\ell }$, we compute the steepest descent direction for all components of ${\widehat{J}}^{\ell }$. Similar to Algorithm 1, we first calculate ${\alpha }_k$ as solution of (4) for ${\mu }_k$. Then, the descent direction is $v_k:=-D{\widehat{J}}^{\ell }{\left({\mu }_k\right)}^{\top }{\alpha }_k$. As a stopping condition, we choose $\sigma ={\parallel \epsilon \parallel }_{\infty }$ and set $a\left({\mu }_k\right)={\mu }_k$ if it holds that

$$\begin{array}{c}\hfill {\parallel D{\widehat{J}}^{\ell }{\left({\mu }_k\right)}^{\top }{\alpha }_k\parallel }_2={\parallel v_k\parallel }_2<\sigma ={\parallel \epsilon \parallel }_{\infty }.\end{array}$$

To generate ${\mathbb{P}}_2^{\ell }$, we use Algorithm 2.

To save computational time during the modified subdivision algorithm, we calculate

$$\begin{array}{c}\hfill \underset{1\le i\le k}{max}{\parallel \nabla {\widehat{J}}_i^N\left(\mu \right)-\nabla {\widehat{J}}_i^{\ell }\left(\mu \right)\parallel }_2\end{array}$$

before the algorithm starts (i.e., in an offline phase) on a training set ${\mathcal{P}}_{\mathsf{train}}$, which approximates ${\mathcal{P}}_{\mathrm{ad}}$ and store these errors. During the subdivision algorithm, we use them to generate the new ${\epsilon }_{i+1}^l$ faster and without calculating the FE solution again.

To see a significant difference between the modified subdivision algorithm and the subdivision algorithm and ${\mathbb{P}}_2^{\ell }$ and ${\mathbb{P}}_1^{\ell }$, we need to have a big error between the gradients. To achieve this, we choose a rough training set

$$\begin{array}{c}\hfill S_{train}=\left\{{(-2+0.51k,-2+0.427j)}^{\top }|(k,j)\in \{0,...,7\}\times \{0,...,9\}\right\}\end{array}$$

with $|S_{train}|=80$. For the Greedy algorithm, we choose the tolerance ${\epsilon }_{tol}=0.06$ and the true error ${\eta }_{\ell }\left(\mu \right):={max}_{1\le i\le k}{\parallel \nabla {\widehat{J}}_i^N\left(\mu \right)-\nabla {\widehat{J}}_i^{\ell }\left(\mu \right)\parallel }_2$ as error indicator, for the DEIM algorithm we choose the tolerance $\overline{\epsilon }=0.5$.

With these settings, we generate an RB-basis with six elements and a DEIM-basis with 18 elements. This leads to the following estimations for the gradients of ${\widehat{J}}^N$ and ${\widehat{J}}^{\ell }$:

$$\begin{array}{cc}\hfill \underset{\mu \in {\mathcal{P}}_{\mathrm{ad}}}{max}{\parallel \nabla {\widehat{J}}_1^N\left(\mu \right)-\nabla {\widehat{J}}_1^{\ell }\left(\mu \right)\parallel }_2& \approx \underset{\mu \in {\mathcal{P}}_{\mathsf{train}}}{max}{\parallel \nabla {\widehat{J}}_1^N\left(\mu \right)-\nabla {\widehat{J}}_1^{\ell }\left(\mu \right)\parallel }_2\approx 0.0491={\epsilon }_1,\hfill \\ \hfill \underset{\mu \in {\mathcal{P}}_{\mathrm{ad}}}{max}{\parallel \nabla {\widehat{J}}_2^N\left(\mu \right)-\nabla {\widehat{J}}_2^{\ell }\left(\mu \right)\parallel }_2& =\underset{\mu \in {\mathcal{P}}_{\mathrm{ad}}}{max}{\parallel \nabla {\widehat{J}}_2\left(\mu \right)-\nabla {\widehat{J}}_2^{\ell }\left(\mu \right)\parallel }_2=0={\epsilon }_2.\hfill \end{array}$$

To generate these estimations, we chose a training set ${\mathcal{P}}_{\mathsf{train}}\subset {\mathcal{P}}_{\mathrm{ad}}$ with 3105 equally distributed test points and generate the error for these points. Thus, we have

$$\begin{array}{c}\hfill {\epsilon }^0=\left(\begin{array}{c}0.0491\\ 0\end{array}\right).\end{array}$$

Now, we test the subdivision and the modified subdivision algorithm with the following conditions. To compute the descent direction, we use the Matlab function fmincon and solve (4) or (7). The algorithm stops when the box size is small enough, which is after 25 iteration steps in our case. In every step, we halve the boxes. We choose five sample points in each box during the first five iteration steps, four sample points in the next five iteration steps, three sample points for the iteration steps ten to 14, two sample points for the next five steps, and one sample points for the last five iteration steps, see Remark 4. Figure 2 shows the generated approximated Pareto sets for the FE-solver after 20 and 25 iteration steps.

Figure 2. Pareto-critical set ${\mathbb{P}}_c$ in iteration step (a) 20 and (b) 25 for the FE-solver.

The results with the subdivision algorithm and RB-DEIM-solver are shown in Figure 3.

Figure 3. (a) ${\mathbb{P}}_1^{\ell }$ and (b) ${\mathbb{P}}_2^{\ell }$ generated with the subdivision algorithm.

The modified subdivision algorithm and RB-DEIM-solver lead to the results plotted in Figure 4.

Figure 4. (a) ${\mathbb{P}}_1^{\ell }$ and (b) ${\mathbb{P}}_2^{\ell }$ generated with the modified subdivision algorithm.

The runtime, number of boxes and number of function and gradient evaluations needed in iteration step 10, 15, 20, and 25 are shown in Table 1, Table 2, Table 3, Table 4 and Table 5. The total runtime and number of function and gradient evaluations needed for the different methods and the speed-up are shown in Table 6.

Table 1. The performance of the subdivision algorithm with the FE-solver and the steepest descent method.

FE
It. Step	Time [s]	# Grad. Solves	# Del. Boxes	# Boxes
10	$370.7$	6872	30	$64$
15	$1077.1$	16786	$166$	$322$
20	$1239.1$	20092	$938$	1388
25	$3364.2$	64721	2567	4749

Table 2. Subdivision algorithm with the steepest descent method.

RB-DEIM
It. Step	Time [s]	# Grad. Solves	# Del. Boxes	# Boxes	$\epsilon$
10	$18.8$	6809	32	$64$	$(0.0491,0)$
15	$42.4$	$14279$	168	$326$	$(0.0491,0)$
20	$44.3$	$13781$	622	$3412$	$(0.0491,0)$
25	$848.6$	196405	149	97551	$(0.0491,0)$

Table 3. Subdivision algorithm with the modified descent direction.

RB-DEIM
It. Step	Time [s]	# Grad. Solves	# Del. Boxes	# Boxes	$\epsilon$
10	$21.0$	6792	31	$63$	$(0.0491,0)$
15	$43.1$	15546	167	$315$	$(0.0491,0)$
20	$38.0$	12655	878	$1438$	$(0.0491,0)$
25	$289.5$	64606	233	30607	$(0.0491,0)$

Table 4. Modified subdivision algorithm with the steepest descent method.

RB-DEIM
It. Step	Time [s]	# Grad. Solves	# Del. Boxes	# Boxes	$\epsilon$
10	$18.5$	6829	32	$64$	$(0.041,0)$
15	$44.1$	16307	170	$318$	$(0.0084,0)$
20	$45.6$	16677	919	$1357$	$(0.0072,0)$
25	$121.2$	29811	560	12230	$(0.0072,0)$

Table 5. Modified subdivision algorithm with the modified descent direction.

RB-DEIM
It. Step	Time [s]	# Grad. Solves	# Del. Boxes	# Boxes	$\epsilon$
10	$18.7$	6805	31	$263$	$(0.041,0)$
15	$44.1$	16459	$168$	$318$	$(0.0084,0)$
20	$49.7$	18266	$937$	1329	$(0.0072,0)$
25	$109.7$	37170	1615	4129	$(0.0043,0)$

Table 6. Total runtime, number of function, and gradient evaluations for the different methods and the speed-up.

Algorithm	Solver	Method	Time [min]	Speed-Up	# Grad. Solves
subdivision	FE	steep. desc.	$495.83$	-	610728
subdivision	RB-DEIM	steep. desc.	$\phantom{1}41.77$	$11.9$	622034
subdivision	RB-DEIM	mod. desc.	$24.77$	$20.0$	390362
mod. subdivision	RB-DEIM	steep. desc.	$21.00$	$23.6$	372551
mod. subdivision	RB-DEIM	mod. desc.	$21.91$	$22.6$	412667

The Greedy algorithm and DEIM together take 14.06 s in the offline phase. To ensure the error $\epsilon$ (cf. (5)) for the gradients on the training set ${\mathcal{P}}_{\mathsf{train}}$, it takes $151.13$ s. It follows from Figure 3 and Figure 4 that ${\mathbb{P}}_2^{\ell }$ is significantly smaller than ${\mathbb{P}}_1^{\ell }$ for the subdivision algorithm as well as for the modified subdivision algorithm. Therefore, we have a much better approximation for ${\mathbb{P}}_c$ if we choose ${\mathbb{P}}_2^{\ell }$ instead of ${\mathbb{P}}_1^{\ell }$. If we compare the two different subdivision algorithms, we notice that with the modified subdivision algorithm we have a better approximation of ${\mathbb{P}}_c$ than with the subdivision algorithm.

The reason for this is that in the modified subdivision algorithm we have a monotonically decreasing sequence ${\epsilon }^l$. In Figure 5, the error between $\nabla {\widehat{J}}_1^N$ and $\nabla {\widehat{J}}_1^{\ell }$ is plotted on the parameter set ${\mathcal{P}}_{\mathrm{ad}}$. The black markers are some points on the Pareto critical set ${\mathbb{P}}_c$. It turns out that the difference between the two gradients is significantly smaller near ${\mathbb{P}}_c$ than in other regions of ${\mathcal{P}}_{\mathrm{ad}}$. Due to this, in the modified subdivision algorithm, the sequence ${\epsilon }^l$ decreases noticeably so that it is useful to update $\epsilon$ after each iteration step. As we have already mentioned, ${\mathbb{P}}_2^{\ell }$ is a better approximation of ${\mathbb{P}}_c$. If ${\mathbb{P}}_2^{\ell }$ is generated by the modified subdivision algorithm, the result is even better. Comparing Figure 2b and Figure 4b, there is no significant difference between the two sets. Therefore, the modified subdivision algorithm yields a good approximation ${\mathbb{P}}_2^{\ell }$ for ${\mathbb{P}}_c$, although the error ${\epsilon }_1$ is not small.

Figure 5. Difference $\parallel \nabla {\widehat{J}}_1^N\left(\mu \right)-\nabla {\widehat{J}}_1^{\ell }{\left(\mu \right)\parallel }_2$ for $\mu \in {\mathcal{P}}_{\mathrm{ad}}$.

Regarding the computational time (cf. Table 1, Table 2, Table 3, Table 4, Table 5 and Table 6), we notice that in the first 20 iterations the four methods take almost the same time. In these iteration steps, the FE-solver takes between 19 and 30 times more time for one iteration step than the four RB-based methods. Only in the subsequent iteration steps a significant difference in the four RB-based methods appears. For these iteration steps, the FE-solver takes between $3.9$ and $30.8$ times as much time as one of the four methods for one iteration step. We notice that the computation of ${\mathbb{P}}_1^{\ell }$ takes around two times as much time as the computation of ${\mathbb{P}}_2^{\ell }$ with the subdivision algorithm. The main reason for this is the much larger number of function and gradient evaluations that we have for ${\mathbb{P}}_1^{\ell }$. This, in turn, can be attributed to the significantly larger number of boxes for ${\mathbb{P}}_1^{\ell }$. If we use the modified subdivision algorithm, the computation of ${\mathbb{P}}_2^{\ell }$ takes slightly more time than the computation of ${\mathbb{P}}_1^{\ell }$. The main reason for this is again the larger number of function and gradient evaluations. Unlike the previous case, this can not be lead back to the number of boxes, but probably to the calculation of the modified descent method, which requires more function and gradient evaluations. Nevertheless, the difference in the computational time is marginal (a factor about $1.04$) for the modified subdivision algorithm. For ${\mathbb{P}}_1^{\ell }$, we notice another behavior: Here, the generation of ${\mathbb{P}}_1^{\ell }$ with the modified subdivision algorithm is 2 times faster than with the subdivision algorithm. This is mainly because of the larger number of function evalutions, which is due to the larger number of boxes. The FE-solver takes approximately $23.6$ times as much time as the computation of ${\mathbb{P}}_1^{\ell }$ and approximately $22.6$ times as much as the time as the computation of ${\mathbb{P}}_2^{\ell }$ with the modified subdivsion. This is another advantage of the modified subdivision algorithm. As we get a better approximation in this algorithm, we have fewer boxes in the iteration steps, and therefore we have a smaller number of function and gradient evaluations than we have for the subdivision algorithm. Finally, we see that the modified subdivision algorithm works better and faster than the subdivision algorithm. As ${\mathbb{P}}_2^{\ell }$ is a tighter approximation of ${\mathbb{P}}_c$, it is better to generate ${\mathbb{P}}_2^{\ell }$ rather than ${\mathbb{P}}_1^{\ell }$.

5. Conclusions

In this article, we present a way to solve multiobjective parameter optimization problems of semilinear elliptic PDEs by combining an RB approach and DEIM with the set-oriented method based on gradient evaluations. To deal with the error introduced by the surrogate model, we derived an additional condition for the descent direction, which allows us to consider the errors for the objective functions independently and derive a superset ${\mathbb{P}}_2^{\ell }$ of the Pareto-critical set ${\mathbb{P}}_c$. To get an even tighter superset, we update these error bounds in our subdivision algorithm after each iteration step. To summarize the numerical results, we first investigated the influence of the error bounds for the gradients of the objective functions. By individually adapting the components of the error bounds, we obtained a tighter covering of the Pareto critical set. When we additionally adjusted the error bounds in each iteration step, the result became even tighter and almost coincided with the exact solution of the MOP (solution with the FE-solver and the steepest descent method). Furthermore, we compared the computational time for each method. The FE-solver needed between $11.9$ times and $23.6$ times more time than the four different RB-based methods we presented in this work. For future work, it could be interesting to improve the results in ([23] Section 3.2.4) and to develop an efficient a posteriori error estimator for the error in the objectives and their gradients, cf., e.g., [32,33,34]. These error bounds can then be used in a weak greedy algorithm and beyond that for the error bounds which are needed in the subdivision algorithm.