Analytical Solutions to Minimum-Norm Problems

Abstract

For $G\in {\mathbb{R}}^{m\times n}$ and $g\in {\mathbb{R}}^m$, the minimization ${min\parallel G\psi -g\parallel }_2$, with $\psi \in {\mathbb{R}}^n$, is known as the Tykhonov regularization. We transport the Tykhonov regularization to an infinite-dimensional setting, that is $min\parallel T\left(h\right)-k\parallel$, where $T:H\to K$ is a continuous linear operator between Hilbert spaces $H,K$ and $h\in H,k\in K$. In order to avoid an unbounded set of solutions for the Tykhonov regularization, we transform the infinite-dimensional Tykhonov regularization into a multiobjective optimization problem: $min\parallel T\left(h\right)-k\parallel \mathrm{and}min\parallel h\parallel$. We call it bounded Tykhonov regularization. A Pareto-optimal solution of the bounded Tykhonov regularization is found. Finally, the bounded Tykhonov regularization is modified to introduce the precise Tykhonov regularization: $min\parallel T\left(h\right)-k\parallel \mathrm{with}\parallel h\parallel =\alpha$. The precise Tykhonov regularization is also optimally solved. All of these mathematical solutions are optimal for the design of Magnetic Resonance Imaging (MRI) coils.

1. Introduction

Optimization problems are among the questions shared between Pure and Applied Mathematics, studied in Operator Theory [1,2,3,4], in Differential Geometry [5,6,7,8], in the Geometry of Banach Spaces [9,10], and in all areas of Medical, Social, and Experimental Sciences [11,12,13,14,15].

By means of optimization problems, it is possible to model with precision many real-life situations, even though, in the case of multiobjective optimization problems, the existence of global and optimal solutions (optimizing all the objective functions at once) is not guaranteed. This is why Pareto-optimal solutions were defined and introduced in the literature of Optimization Theory. In an informal way, a Pareto-optimal solution is a feasible solution satisfying that if there exists another feasible solution better optimizing one objective function, then the latter has to be less optimal in another objective function.

As shown in [14,16,17,18,19], the design of optimal MRI coils is modeled by means of a particular case of optimization problems called minimum-norm problems [20], such as

$$\left\{\begin{array}{c}{min\parallel \psi \parallel }_2,\hfill \\ G\psi =g,\hfill \end{array}\right.\left\{\begin{array}{c}{min\parallel \psi \parallel }_2,\hfill \\ {\parallel G\psi -g\parallel }_{\infty }\le D,\hfill \end{array}\right.\mathrm{and}\left\{\begin{array}{c}{min\parallel G\psi -g\parallel }_2,\hfill \\ \psi \in {\mathbb{R}}^n,\hfill \end{array}\right.$$

for $G\in {\mathbb{R}}^{m\times n}$, $\psi \in {\mathbb{R}}^n$, $g\in {\mathbb{R}}^m$, and $D\ge 0$. Notice that the last one of the three above problems is precisely the finite-dimensional Tykhonov regularization. A theoretical treatment in the framework of Operator Theory and Functional Analysis will be given to the above problems, transporting them to an infinite-dimensional setting, as well as a MATLAB encoding for their finite-dimensional version in Appendices A–D.

We also introduce in the literature of Optimization Theory the following multiobjective optimization problem:

$$\left\{\begin{array}{c}min\parallel T\left(h\right)-k\parallel ,\hfill \\ min\parallel h\parallel ,\hfill \end{array}\right.$$

where $T:H\to K$ is a continuous linear operator between Hilbert spaces $H,K$ and $h\in H,k\in K$, which we call bounded Tykhonov regularization. We provide a nontrivial Pareto-optimal solution of the bounded Tykhonov regularization. Sometimes, the  bounded Tykhonov regularization might produce a nontrivial Pareto-optimal solution of an excessively small norm. This is why we introduce what we name as the precise Tykhonov regularization:

$$\left\{\begin{array}{c}min\parallel T\left(h\right)-k\parallel ,\hfill \\ \parallel h\parallel =\alpha ,\hfill \end{array}\right.$$

which we fully and optimally solve.

2. Materials and Methods

In this Methodology Section, we properly define the optimization problems that we will deal with. We will also gather all the necessary concepts, notions, techniques, and results needed to accomplish our analytical solutions for the previously mentioned optimization problems.

2.1. Mathematical Formulation of the Optimization Problems

The optimization problems that we will deal with in this manuscript are described next. As we mentioned before, these three problems arise from the optimal design of MRI coils.

Problem 1.

Let $G\in {\mathbb{R}}^{m\times n}$ and $g\in {\mathbb{R}}^m$ in the range of G. Solve

$$\left\{\begin{array}{c}{min\parallel \psi \parallel }_2,\hfill \\ G\psi =g,\hfill \end{array}\right.$$

(1)

for $\psi \in {\mathbb{R}}^n$.

Observe that, under the settings of Problem 1, g must be in the range of G, that is there must exist at least one $\psi \in {\mathbb{R}}^n$ for which $G\psi =g$. Otherwise, the feasible region of Problem 1 is empty. Furthermore, if $g=0$, then 0 is trivially the unique solution of Problem 1. The infinite-dimensional or abstract version of Problem 1 is displayed next.

Problem 2.

Let $X,Y$ be Banach Spaces, $T:X\to Y$ a continuous linear operator, and $y\in T\left(X\right)$. Solve

$$\left\{\begin{array}{c}min\parallel x\parallel ,\hfill \\ T\left(x\right)=y,\hfill \end{array}\right.$$

(2)

for $x\in X$.

Under the settings of Problem 2, $x=0$ is the (unique) solution of Problem 2 if and only if $y=0$.

Problem 3.

Let $G\in {\mathbb{R}}^{m\times n}$, $g\in {\mathbb{R}}^m\backslash \left\{0\right\}$, and ${0<D<\parallel g\parallel }_{\infty }$. Solve

$$\left\{\begin{array}{c}{min\parallel \psi \parallel }_2,\hfill \\ {\parallel G\psi -g\parallel }_{\infty }\le D,\hfill \end{array}\right.$$

(3)

for $\psi \in {\mathbb{R}}^n$.

Notice that, under the settings of Problem 3, g does not necessarily need to be in the range of G. However, if the condition ${0<D<\parallel g\parallel }_{\infty }$ is not imposed, then $0\le {\parallel g\parallel }_{\infty }\le D$ allows 0 to be the unique solution of Problem 3. The infinite-dimensional or abstract version of Problem 3 is as follows.

Problem 4.

Let $X,Y$ be Banach Spaces, $T:X\to Y$ a continuous linear operator, $y\in Y\backslash \left\{0\right\}$, and $0<D<\parallel y\parallel$. Solve

$$\left\{\begin{array}{c}min\parallel x\parallel ,\hfill \\ \parallel T\left(x\right)-y\parallel \le D,\hfill \end{array}\right.$$

(4)

for $x\in X$.

Under the settings of Problem 4, the reason for D to lie in the open interval $(0,\parallel y\parallel )$ is again to avoid the trivial solution $x=0$. Indeed, $x=0$ is the (unique) solution of Problem 4 if and only if $D\ge \parallel y\parallel$. The following technical lemma ensures that any solution of Problem 4 must lie in the boundary of the feasible region. This lemma will be useful later on.

Lemma 1.

Let $X,Y$ be Banach Spaces, $T:X\to Y$ a continuous linear operator, $y\in Y\backslash \left\{0\right\}$, and $0<D<\parallel y\parallel$. If $x\in X$ is a solution of Problem 4, then $\parallel T\left(x\right)-y\parallel =D$.

Proof. 

If $\parallel T\left(x\right)-y\parallel <D$, then we can find $\frac{\parallel y\parallel -D}{\parallel y\parallel -\parallel T\left(x\right)-y\parallel }\le \delta <1$, which implies that $\parallel T\left(\delta x\right)-y\parallel \le \parallel T\left(\delta x\right)-\delta y\parallel +\parallel \delta y-y\parallel =\delta \parallel T\left(x\right)-y\parallel +(1-\delta )\parallel y\parallel \le D$. Then, $\delta x$ is in the feasible region of Problem 4, reaching the contradiction that $\parallel \delta x\parallel =\delta \parallel x\parallel <\parallel x\parallel \le \parallel \delta x\parallel$.    □

The third problem that we will deal with is the Tykhonov regularization.

Problem 5

(Finite-dimensional Tykhonov regularization). Let $G\in {\mathbb{R}}^{m\times n}$ and $g\in {\mathbb{R}}^m$. Solve

$$\left\{\begin{array}{c}{min\parallel G\psi -g\parallel }_2,\hfill \\ \psi \in {\mathbb{R}}^n.\hfill \end{array}\right.$$

(5)

The infinite-dimensional or abstract version of Problem 5 follows next.

Problem 6

(Infinite-dimensional Tykhonov regularization). Let $X,Y$ be Banach Spaces, $T:X\to Y$ a continuous linear operator, and $y\in Y$. Solve

$$\left\{\begin{array}{c}min\parallel T\left(x\right)-y\parallel ,\hfill \\ x\in X.\hfill \end{array}\right.$$

(6)

Notice that, under the settings of Problem 6, when $y=0$, we obtain the trivial set of solutions given by $ker\left(T\right)$.

2.2. Supporting Vectors

If $X,Y$ are Banach Spaces and $T:X\to Y$ is a continuous linear operator, then the operator norm of T is defined as

$$\parallel T\parallel :=\underset{\parallel x\parallel \le 1}{sup}\parallel T\left(x\right)\parallel .$$

This norm turns the vector space of continuous linear operators from X to Y, $\mathcal{C}\mathcal{L}(X,Y)$, into a Banach Space. When $X=Y$, we will simply denote it by $\mathcal{C}\mathcal{L}\left(X\right)$. When $Y=\mathbb{K}$ ($\mathbb{R}$ or ), we will simply denote it by $X^{*}$.

The concept of the supporting vector was formerly introduced for the first time in [1], although it appeared implicitly and scattered throughout the literature of Banach Space Theory, as for instance, in [3,4,9,10].

Definition 1

(Supporting vector). Let $X,Y$ be Banach Spaces. Let $T:X\to Y$ be a continuous linear operator. The set of supporting vectors of T is defined as

$$\mathrm{suppv}\left(T\right):=\{x\in X:\parallel x\parallel =1\mathrm{and}\parallel T(x)\parallel =\parallel T\parallel \}.$$

We refer the reader to [2,21,22] for a topological and geometrical study of the set of supporting vectors of a continuous linear operator. Supporting vectors have been successfully applied to solve multiobjective optimization problems that typically arise in Bioengineering, Physics, and Statistics [14,15,23,24,25], improving considerably the results obtained by means of other techniques, such as Heuristic methods [16,18,19].

Definition 2

(1-Supporting vector). Let X be a Banach Space. Let $f\in X^{*}$ be a continuous linear functional. The set of 1-supporting vectors of f is defined as

$${\mathrm{suppv}}_{\mathtt{1}}\left(f\right):=\{x\in X:\parallel x\parallel =1\mathrm{and}f\left(x\right)=\parallel f\parallel \}.$$

The 1-supporting vectors are special cases of supporting vectors, that is ${\mathrm{suppv}}_{\mathtt{1}}\left(f\right)\subseteq \mathrm{suppv}\left(f\right)$. We will strongly rely on 1-supporting vectors later on. A standard geometrical property of 1-supporting vectors is shown in the next remark.

Remark 1.

Let X be a Banach Space. Let $f\in X^{*}\backslash \left\{0\right\}$. If $x,y\in {\mathrm{suppv}}_{\mathtt{1}}\left(f\right)$, then $tx+(1-t)y\in {\mathrm{suppv}}_{\mathtt{1}}\left(f\right)$ for all $t\in [0,1]$. In other words, ${\mathrm{suppv}}_{\mathtt{1}}\left(f\right)$ is a convex subset of the unit sphere ${\mathsf{S}}_X$ of X.

2.3. Riesz Representation Theorem on Hilbert Spaces

The Riesz Representation Theorem is one of the most important results in Functional Analysis and is crucial for working with self-adjoint operators on Hilbert spaces.

Theorem 1

(Riesz Representation Theorem). Let H be a Hilbert space. The dual map of H,

$$\begin{array}{cccc}\hfill J_H:& \hfill H& \to & H^{*}\hfill \\ \hfill & \hfill h& \mapsto & J_H\left(h\right):=h^{*}=(•\left|h\right),\hfill \end{array}$$

(7)

where

$$\begin{array}{cccc}\hfill J_H\left(h\right)=h^{*}=(•\left|h\right):& \hfill H& \to & \mathbb{K}\hfill \\ & \hfill k& \mapsto & J_H\left(h\right)\left(k\right)=h^{*}\left(k\right)=(k\left|h\right),\hfill \end{array}$$

(8)

is a surjective linear isometry between H and $H^{*}$.

In the frame of the Geometry of Banach Spaces, $J_H$ is called the duality mapping. In Quantum Mechanics, the dual map $J_H$ has a different mention and notation. Under the settings of the Riesz Representation Theorem and by relying on certain techniques of the Geometry of Banach Spaces and on Remark 1, it can be proven that if $h\in H\backslash \left\{0\right\}$, then $\frac{h}{\parallel h\parallel }$ is the only 1-supporting vector of $h^{*}$, that is ${\mathrm{suppv}}_{\mathtt{1}}\left(h^{*}\right)=\left\{\frac{h}{\parallel h\parallel }\right\}$.

Let H be a Hilbert space. For every closed subspace M of H, the orthogonal subspace of M is denoted by $M^{\perp }$ and the orthogonal projection of H onto M is denoted as $p_M$. Notice that $H=M{\oplus }_2{M}^{\perp }$, in other words, for all $x\in H$,

$$x=p_M\left(x\right)+p_{M^{\perp }}\left(x\right)\mathrm{and}{\parallel x\parallel }^2={∥p_M\left(x\right)∥}^2+{∥p_{M^{\perp }}\left(x\right)∥}^2.$$

(9)

If $H,K$ are Hilbert spaces and $T:H\to K$ is a continuous linear operator, then there exists a unique continuous linear operator $T^{*}:K\to H$ such that $(T\left(h\right)\left|k\right)=\left(h|T^{*}\left(k\right)\right)$ for all $h\in H$ and all $k\in K$. This operator $T^{*}$ is called the adjoint of T. The following technical lemma is well known in the literature of Functional Analysis and Operator Theory, and it will be used later on.

Lemma 2.

Let $H,K$ be Hilbert spaces. Let $T:H\to K$ be a continuous linear operator. Then, $T{\left(H\right)}^{\perp }=ker\left(T^{*}\right)$ and $T\left(H\right)=ker{\left(T^{*}\right)}^{\perp }$.

The finite-dimensional Hilbert spaces, which are involved in the finite-dimensional problems previously mentioned, will be denoted by ${\ell }_2^n:=\left({\mathbb{R}}^n,{\parallel ·\parallel }_2\right)$, where ${\parallel ·\parallel }_2$ clearly denotes the Euclidean norm.

3. Results

This section is aimed at providing analytical solutions for Problems 2 and 4, which will automatically work for Problems 1 and 3, respectively, since these last two problems are particular cases of the first two.

3.1. Analytical Solution of Problems 1 and 2 in the Hilbert Space Context

Problem 2 will be actually tackled, and solved completely, in the Hilbert space context. The reformulation of Problem 2 in the previously mentioned setting follows next.

Problem 7.

Let $H,K$ be Hilbert spaces, $T:H\to K$ a continuous linear operator, and $k\in T\left(H\right)$. Solve

$$\left\{\begin{array}{c}min\parallel h\parallel ,\hfill \\ T\left(h\right)=k,\hfill \end{array}\right.$$

(10)

for $h\in H$.

Observe that Problem 1 is still a particular case of Problem 7, which is itself a particular case of Problem 2.

Lemma 3.

Let H be a Hilbert space. Let M be a closed subspace of H. For every $x\in X$,

$$(x+M)\cap M^{\perp }=\left\{p_{M^{\perp }}\left(x\right)\right\}\mathit{and}min\{\parallel x+m\parallel :m\in M\}=∥p_{M^{\perp }}\left(x\right)∥.$$

Proof. 

We will show first that $(x+M)\cap M^{\perp }=\left\{p_{M^{\perp }}\left(x\right)\right\}$. Indeed, on the one hand, it is clear that $p_{M^{\perp }}\left(x\right)\in M^{\perp }$ and, by (9), $p_{M^{\perp }}\left(x\right)=x+\left(-p_M\left(x\right)\right)\in x+M$, so $(x+M)\cap M^{\perp }\supseteq \left\{p_{M^{\perp }}\left(x\right)\right\}$. On the other hand, take arbitrary elements $m\in M$ and $m^{\perp }\in M^{\perp }$ such that $x+m=m^{\perp }$. By using again (9),

$$m^{\perp }=x+m=p_M\left(x\right)+p_{M^{\perp }}\left(x\right)+m,$$

meaning that

$$m^{\perp }-p_{M^{\perp }}\left(x\right)=p_M\left(x\right)-m\in M^{\perp }\cap M=\left\{0\right\}.$$

As a consequence, $m^{\perp }=p_{M^{\perp }}\left(x\right)$, and so, $x+m=m^{\perp }=p_{M^{\perp }}\left(x\right)$. This proves the inclusion $(x+M)\cap M^{\perp }\subseteq \left\{p_{M^{\perp }}\left(x\right)\right\}$. Let us finally prove that $min\{\parallel x+m\parallel :m\in M\}=∥p_{M^{\perp }}\left(x\right)∥.$ Fix an arbitrary $m\in M$. By virtue of (9), note that

$$\begin{array}{ccc}\hfill {\parallel x+m\parallel }^2& =& {∥p_M\left(x\right)+p_{M^{\perp }}\left(x\right)+m∥}^2\hfill \\ & =& {∥p_M(x+m)+p_{M^{\perp }}\left(x\right)∥}^2\hfill \\ & =& {∥p_M(x+m)∥}^2+{∥p_{M^{\perp }}\left(x\right)∥}^2\hfill \\ & \ge & {∥p_{M^{\perp }}\left(x\right)∥}^2.\hfill \end{array}$$

Therefore,

$$min\{\parallel x+m\parallel :m\in M\}\ge ∥p_{M^{\perp }}\left(x\right)∥.$$

Since $p_{M^{\perp }}\left(x\right)\in x+M$ as we have just proven first, we finally conclude that

$$min\{\parallel x+m\parallel :m\in M\}=∥p_{M^{\perp }}\left(x\right)∥.$$

   □

Remark 2.

Under the settings of Lemma 3, for every $y\in x+M$, it is clear that $y+M=x+M$ and $p_{M^{\perp }}\left(x\right)=p_{M^{\perp }}\left(y\right)$.

The following theorem solves Problem 7 completely.

Theorem 2.

Let $H,K$ be Hilbert spaces. Let $T:H\to K$ be a continuous linear operator. For every $k_0\in T\left(H\right)$ and every $h_0\in T^{-1}\left(k_0\right)$, we have the following:

1. 

$min\left\{\parallel h\parallel :h\in H,T\left(h\right)=k_0\right\}=∥p_{ker{\left(T\right)}^{\perp }}\left(h_0\right)∥.$

2. 

The above min is attained at $p_{ker{\left(T\right)}^{\perp }}\left(h_0\right)\in T^{-1}\left(k_0\right)$.

3. 

If $h_1\in T^{-1}\left(k_0\right)$, then $p_{ker{\left(T\right)}^{\perp }}\left(h_0\right)=p_{ker{\left(T\right)}^{\perp }}\left(h_1\right)$.

Proof. 

In the first place, observe that

$$\left\{h\in H:T\left(h\right)=k_0\right\}=T^{-1}\left(k_0\right)=h_0+ker\left(T\right).$$

By relying on Lemma 3,

$$min\left\{\parallel h\parallel :h\in H,T\left(h\right)=k_0\right\}=min\left\{∥h_0+h^{\prime }∥:h^{\prime }\in ker\left(T\right)\right\}=∥p_{ker{\left(T\right)}^{\perp }}\left(h_0\right)∥.$$

Finally, by taking into consideration Lemma 3 together with Remark 2, we see that $p_{ker{\left(T\right)}^{\perp }}\left(h_0\right)\in T^{-1}\left(k_0\right)$ and $p_{ker{\left(T\right)}^{\perp }}\left(h_0\right)=p_{ker{\left(T\right)}^{\perp }}\left(h_1\right)$ for each $h_1\in T^{-1}\left(k_0\right)$.    □

Now, we are in the right position to provide a full solution to Problem 1.

Corollary 1.

Let $G\in {\mathbb{R}}^{m\times n}$ and $g\in {\mathbb{R}}^m$ be in the range of G. The solution of Problem 1 is given by ${∥p_{ker{\left(G\right)}^{\perp }}\left({\psi }_0\right)∥}_2$ for any ${\psi }_0\in {\mathbb{R}}^n$ such that $G{\psi }_0=g$, and it is attained at $p_{ker{\left(G\right)}^{\perp }}\left({\psi }_0\right)$.

Proof. 

It only suffices to call on Theorem 2 by taking $H:={\ell }_2^n$, $K:={\ell }_2^m$, $T:=G$, $k_0:=g$, and ${\psi }_0:=h_0$.    □

A MATLAB encoding for Corollary 1 is available in Appendix A.

3.2. Analytical Solution of Problem 7 When $K:=\mathbb{K}$

If we take $K:=\mathbb{K}$ in Problem 7, then its solution can be also computed in terms of 1-supporting vectors.

Theorem 3.

Let H be a Hilbert space. Let $h_0^{*}\in H^{*}\backslash \left\{0\right\}$. For every $\lambda \in \mathbb{K}$,

$$min\left\{\parallel h\parallel :h\in H,h_0^{*}\left(h\right)=\lambda \right\}=\frac{\left|\lambda \right|}{∥h_0∥}.$$

Even more, the previous min is attained at $\frac{\lambda }{∥h_0∥}\frac{h_0}{\parallel h_0\parallel }$.

Proof. 

First off, note that, for every $h\in H$ with $h_0^{*}\left(h\right)=\lambda$,

$$\left|\lambda \right|=\left|h_0^{*}\left(h\right)\right|\le ∥h_0^{*}∥\parallel h\parallel =\parallel h_0\parallel \parallel h\parallel ,$$

meaning that

$$\parallel h\parallel \ge \frac{\left|\lambda \right|}{∥h_0∥}.$$

This proves the inequality:

$$min\left\{\parallel h\parallel :h\in H,h_0^{*}\left(h\right)=\lambda \right\}\ge \frac{\left|\lambda \right|}{∥h_0∥}.$$

In order to prove the reverse inequality, we will make use of $\frac{\lambda }{{∥h_0∥}^2}{h}_0$. Observe that

$$h_0^{*}\left(\frac{\lambda }{{∥h_0∥}^2}{h}_0\right)=\lambda \frac{h_0^{*}\left(h_0\right)}{\parallel h_0{\parallel }^2}=\lambda \frac{\parallel h_0{\parallel }^2}{\parallel h_0{\parallel }^2}=\lambda .$$

Therefore,

$$\frac{\lambda }{{∥h_0∥}^2}{h}_0\in \left\{h\in H:h_0^{*}\left(h\right)=\lambda \right\}.$$

Next,

$$∥\frac{\lambda }{{∥h_0∥}^2}{h}_0∥=\frac{\left|\lambda \right|}{{∥h_0∥}^2}\parallel h_0\parallel =\frac{\left|\lambda \right|}{∥h_0∥}.$$

This finally shows that

$$min\left\{\parallel h\parallel :h\in H,h_0^{*}\left(h\right)=\lambda \right\}=\frac{\left|\lambda \right|}{∥h_0∥}$$

and it is attained at $\frac{\lambda }{{∥h_0∥}^2}{h}_0$.    □

As an immediate corollary, if we take $m=1$ in Problem 1, then its solution can be also computed in terms of 1-supporting vectors.

Corollary 2.

Let $G\in {\mathbb{R}}^{1\times n}$, $G\ne 0$, and $g\in \mathbb{R}$. The solution of Problem 1 is given by $\frac{\left|g\right|}{∥G^t∥}$, and it is attained at $\psi :=\frac{g}{∥G^t∥}\frac{G^t}{∥G^t∥}$.

Proof. 

It only suffices to call on Theorem 3 by taking $H:={\ell }_2^n$, $h_0^{*}:=G$, $\lambda :=g$, and $h_0:=G^t$.    □

A MATLAB encoding for Corollary 2 is available in Appendix B.

3.3. Partial Solution of Problem 3

A particular version of Problem 3 was partially solved in [20] (Corollary 13). Here, we will follow a completely different approach. Before tackling Problem 3, we will first solve particular cases of it.

Problem 8.

Let X be a Banach Space, $f\in X^{*}\backslash \left\{0\right\}$, and $0<c\le d$. Solve

$$\left\{\begin{array}{c}min\parallel x\parallel ,\hfill \\ c\le f\left(x\right)\le d.\hfill \end{array}\right.$$

(11)

We will first solve Problem 8 by relying on 1-supporting vectors.

Lemma 4.

Let X be a Banach Space, $f\in X^{*}\backslash \left\{0\right\}$, and $0<c\le d$. The set of optimal solutions of Problem 8 is given by $\mathrm{sol}\left(11\right)=\left\{\frac{cx}{\parallel f\parallel }:x\in {\mathrm{suppv}}_{\mathtt{1}}\left(f\right)\right\}.$ In particular, $\mathrm{sol}\left(11\right)\ne ⌀$ if and only if ${\mathrm{suppv}}_{\mathtt{1}}\left(f\right)\ne ⌀$.

Proof. 

⊆

Fix an arbitrary $y\in \mathrm{sol}\left(11\right)$. We will show that $x:=\frac{\parallel f\parallel }{c}y\in {\mathrm{suppv}}_{\mathtt{1}}\left(f\right)$. For every $z\in {\mathsf{B}}_X\backslash ker\left(f\right)$, $\frac{c}{f\left(z\right)}z$ is in the feasible region of (11) since $f\left(\frac{c}{f\left(z\right)}z\right)=c$; therefore,

$$\parallel y\parallel \le ∥\frac{c}{f\left(z\right)}z∥=\frac{c}{\left|f\right(z\left)\right|}\parallel z\parallel .$$

(12)

Now, if we take a sequence ${\left(z_n\right)}_{n\in \mathbb{N}}\subseteq {\mathsf{S}}_X\backslash ker\left(f\right)$ such that $\left|f\left(z_n\right)\right|\to \parallel f\parallel$, we obtain from (12) that $\parallel y\parallel \le \frac{c}{\parallel f\parallel }$. On the other hand, $c\le f\left(y\right)\le \parallel f\parallel \parallel y\parallel$, that is $\frac{c}{\parallel f\parallel }\le \parallel y\parallel$. As a consequence, $\parallel y\parallel =\frac{c}{\parallel f\parallel }$, meaning that $\parallel x\parallel =1$. Finally, notice that

$$\parallel f\parallel \ge f\left(x\right)=f\left(\frac{\parallel f\parallel }{c}y\right)=\frac{\parallel f\parallel }{c}f\left(y\right)\ge \frac{\parallel f\parallel }{c}c=\parallel f\parallel ,$$

which implies that $f\left(x\right)=\parallel f\parallel$, hence $x\in {\mathrm{suppv}}_{\mathtt{1}}\left(f\right)$.

⊇

Take any $x\in {\mathrm{suppv}}_{\mathtt{1}}\left(f\right)$. In the first place, $f\left(\frac{cx}{\parallel f\parallel }\right)=c$, so $\frac{cx}{\parallel f\parallel }$ is in the feasible region of problem (11), that is $\frac{cx}{\parallel f\parallel }$ is a feasible solution. Next, take y as another feasible solution of (11). Then, $c\le f\left(y\right)\le \parallel f\parallel \parallel y\parallel$; hence,

$$\parallel y\parallel \ge \frac{c}{\parallel f\parallel }=∥\frac{cx}{\parallel f\parallel }∥.$$

This shows that $\frac{cx}{\parallel f\parallel }\in \mathrm{sol}\left(11\right)$; in other words, $\frac{cx}{\parallel f\parallel }$ is an optimal solution

   □

Notice that the feasible region of Problem 8, $c\le f\left(x\right)\le d$, can be rewritten as $|f\left(x\right)-\frac{c+d}{2}|\le \frac{d-c}{2}$; hence, Problem 8 is of the same form as Problem 3 whenever $c<d$. In the case that $c=d$, then Problem 8 is a particular case of Problem 1. In fact, Problem 3 can be rewritten as follows.

Problem 9.

Let $G\in {\mathbb{R}}^{m\times n}$, $g\in {\mathbb{R}}^m\backslash \left\{0\right\}$, and ${0<D<\parallel g\parallel }_{\infty }$. Solve

$$\left\{\begin{array}{c}{min\parallel \psi \parallel }_2\hfill \\ {\parallel G\psi -g\parallel }_{\infty }\le D\hfill \end{array}\right.\iff \left\{\begin{array}{c}{min\parallel \psi \parallel }_2\hfill \\ |G_1\psi -g_1|\le D\hfill \\ ⋮\hfill \\ |G_m\psi -g_m|\le D\hfill \end{array}\right.\iff \left\{\begin{array}{c}{min\parallel \psi \parallel }_2\hfill \\ g_1-D\le G_1\psi \le g_1+D\hfill \\ ⋮\hfill \\ g_m-D\le G_m\psi \le g_m+D\hfill \end{array}\right.$$

(13)

for $\psi \in {\mathbb{R}}^n$, where $g={(g_1,\cdots ,g_m)}^t$ and $G_i$ is the $i^{th}$ row vector of G for $i=1,\cdots ,m$.

If, under the settings of Problem 9, we assume that $g_1=\cdots =g_m>0$, which is consistent with the design of optimal MRI coils according to [14,16,18,19], then $g_i-D={\parallel g\parallel }_{\infty }-D>0$ for all $i=1,\cdots ,m$. As a consequence, $g_i-D\le G_i\psi \le g_i+D$ is of the same form as the feasible region of Problem 11. Observe that, under this assumption, $G_i\ne 0$ for all $i=1,\cdots ,m$. In this situation, the infinite-dimensional generalization of Problem 9 is given as follows.

Problem 10.

Let X be a Banach Space, $f_i\in X^{*}\backslash \left\{0\right\}$, $i=1,\cdots ,m$, and $0<c\le d$. Solve

$$\left\{\begin{array}{c}min\parallel x\parallel ,\hfill \\ c\le f_1\left(x\right)\le d\hfill \\ ⋮\hfill \\ c\le f_m\left(x\right)\le d,\hfill \end{array}\right.$$

(14)

for $x\in X$.

Notice that, in the case $c<d$, Problem 10 is a particular case of Problem 4 since one can define the following continuous linear operator:

$$\begin{array}{cccc}\hfill T:& \hfill X& \to & {\mathbb{R}}^m\hfill \\ & \hfill x& \mapsto & T\left(x\right):=\left(f_1\left(x\right),\cdots ,f_m\left(x\right)\right).\hfill \end{array}$$

(15)

Then, the feasible region of Problem 10 is precisely

$$\{x\in X:c\le f_i\left(x\right)\le d\forall i=1,\cdots ,m\}=\left\{x\in X:{∥T\left(x\right)-g∥}_{\infty }\le D\right\},$$

where $g:=\left(\frac{c+d}{2},\stackrel{m}{\cdots },\frac{c+d}{2}\right)$ and $D:=\frac{d-c}{2}$. If $c=d$, then by using the same operator T given in Equation (15), it can be seen that Problem 10 is a particular case of Problem 2. We will strongly rely on Lemma 4 to approach the optimal solutions of Problem 10.

Theorem 4.

Let X be a Banach Space, $f_i\in X^{*}\backslash \left\{0\right\}$, $i=1,\cdots ,m$, and $0<c\le d$. If there exists $i\in \{1,\cdots ,m\}$ and $x\in {\mathrm{suppv}}_{\mathtt{1}}\left(f_i\right)$ such that $\frac{c}{\parallel f_i\parallel }x\in f_k^{-1}\left([c,d]\right)$ for all $k\in \{1,\cdots ,m\}\backslash \left\{i\right\}$, then $\frac{c}{\parallel f_i\parallel }x$ is an optimal solution of Problem 10.

Proof. 

In the first place, notice that $\frac{c}{\parallel f_i\parallel }x$ is a feasible solution of Problem 10, that is it belongs to the feasible region simply because

$$c\le f_k\left(\frac{c}{\parallel f_i\parallel }x\right)\le d$$

for all $k=1,\cdots ,m$. Let $z\in X$ be another feasible solution of Problem 10. In particular, $c\le f_i\left(z\right)\le d$; therefore, if we consider Problem 8 for $f_i$, we have that $\frac{c}{\parallel f_i\parallel }x$ is an optimal solution of such a problem in view of Lemma 4. As a consequence, $∥\frac{c}{\parallel f_i\parallel }x∥\le \parallel z\parallel$. Then, we claim that $\frac{c}{\parallel f_i\parallel }x$ is an optimal solution of Problem 10.    □

3.4. Analytical Solution of Problems 5 and 6 in the Hilbert Space Context

The solution of the finite-dimensional Tykhonov regularization is well known in the literature of Optimization Theory. Here, we present a fine argument to solve the infinite-dimensional Tykhonov regularization in the Hilbert context, whose formulation follows.

Problem 11

(Infinite-dimensional Tykhonov regularization). Let $H,K$ be Hilbert spaces, $T:H\to K$ a continuous linear operator, and $k\in K$. Solve

$$\left\{\begin{array}{c}min\parallel T\left(h\right)-k\parallel ,\hfill \\ h\in H.\hfill \end{array}\right.$$

(16)

We will rely on the basic techniques of Hilbert spaces and Operator Theory.

Proposition 1.

Let $H,K$ be Hilbert spaces, $T:H\to K$ a continuous linear operator, and $k\in K$. Then:

1. 

If $k\in T\left(H\right)$, then $min\{\parallel T(h)-k\parallel :h\in H\}=0$, and it is attained at any element of $T^{-1}\left(k\right)$.

2. 

If T has dense range, then $inf\{\parallel T(h)-k\parallel :h\in H\}=0$. Hence, Problem 11 has a solution if and only if $k\in T\left(H\right)$.

Proof. 

• This is a simple and trivial exercise.
• Suppose that T has a dense range, that is the closure of $T\left(H\right)$ is K. There exists a sequence ${\left(h_n\right)}_{n\in \mathbb{N}}$ such that ${\left(T\left(h_n\right)\right)}_{n\in \mathbb{N}}$ converges to k. This means that $inf\{\parallel T(h)-k\parallel :h\in H\}=0$. Next, if Problem 11 has a solution $h_0\in H$, then $\parallel T\left(h_0\right)-k\parallel =min\{\parallel T\left(h\right)-k\parallel :h\in H\}=inf\{\parallel T\left(h\right)-k\parallel :h\in H\}=0$, which implies that $k=T\left(h_0\right)\in T\left(H\right)$. Conversely, suppose that $k\in T\left(H\right)$, that is there exists $h_0\in H$ with $T\left(h_0\right)=k$. Then, $\parallel T\left(h_0\right)-k\parallel =0$, so it is clear that $inf\{\parallel T(h)-k\parallel :h\in H\}=min\{\parallel T(h)-k\parallel :h\in H\}=0$.

   □

The following theorem is a refinement of Lemma 3.

Theorem 5.

Let H be a Hilbert space. Let M be a closed subspace of H. Let $h\in H$. Then, $min\{\parallel h-m\parallel :m\in M\}=∥p_{M^{\perp }}\left(h\right)∥$. Even more, the previous min is attained at $p_M\left(h\right)$.

Proof. 

We can write $h=p_M\left(h\right)+p_{M^{\perp }}\left(h\right)$. For every $m\in M$,

$${\parallel h-m\parallel }^2={∥p_M\left(h\right)+p_{M^{\perp }}\left(h\right)-m∥}^2={∥p_M\left(h\right)-m∥}^2+{∥p_{M^{\perp }}\left(h\right)∥}^2\ge {∥p_{M^{\perp }}\left(h\right)∥}^2.$$

This shows that $min\{\parallel h-m\parallel :m\in M\}\ge ∥p_{M^{\perp }}\left(h\right)∥$. Finally, notice that $p_M\left(h\right)\in M$ and $\parallel h-p_M\left(h\right)\parallel =∥p_{M^{\perp }}\left(h\right)∥.$ This shows that $min\{\parallel h-m\parallel :m\in M\}=∥p_{M^{\perp }}\left(h\right)∥$, and the min is attained at $p_M\left(h\right)$.    □

As an immediate consequence of Theorem 5, we obtain the following corollary, which fully solves Problem 11.

Corollary 3.

Let $H,K$ be Hilbert spaces. Let $T:H\to K$ be a continuous linear operator of closed range. For every $k_0\in K$, we have the following:

1. 

$min\left\{\parallel T\left(h\right)-k_0\parallel :h\in H\right\}=∥p_{T{\left(H\right)}^{\perp }}\left(k_0\right)∥.$

2. 

The above min is attained at any element of $T^{-1}\left(p_{T\left(H\right)}\left(k_0\right)\right)$.

3. 

$\arg min\left\{\parallel T\left(h\right)-k_0\parallel :h\in H\right\}$ is bounded if and only if $ker\left(T\right)=\left\{0\right\}$.

Proof. 

The first two items are a direct application of Theorem 5, so let us simply take care of the third item. Note that $\arg min\left\{\parallel T\left(h\right)-k_0\parallel :h\in H\right\}=T^{-1}\left(p_{T\left(H\right)}\left(k_0\right)\right)=h_0+ker\left(T\right)$ for any $h_0\in T^{-1}\left(p_{T\left(H\right)}\left(k_0\right)\right)$. As a consequence, $\arg min\left\{\parallel T\left(h\right)-k_0\parallel :h\in H\right\}$ is bounded if and only if $ker\left(T\right)=\left\{0\right\}$.    □

4. Discussion

We discuss in this section several aspects of the obtained results.

4.1. Bounded Tykhonov Regularization

The bounded Tykhonov regularization is a novel concept conceived of in this manuscript describing a different way to tackle the classical Tykhonov regularization in an original manner that allows designing efficient MRI coils. The bounded Tykhonov regularization is described as the following multiobjective optimization problem.

Problem 12

(Finite-dimensional bounded Tykhonov regularization). Let $G\in {\mathbb{R}}^{m\times n}$ and $g\in {\mathbb{R}}^m$. Solve

$$\left\{\begin{array}{c}{min\parallel G\psi -g\parallel }_2,\hfill \\ {min\parallel \psi \parallel }_2,\hfill \end{array}\right.$$

(17)

for $\psi \in {\mathbb{R}}^n$.

The infinite-dimensional version of the bounded Tykhonov regularization follows now.

Problem 13

(Infinite-dimensional bounded Tykhonov regularization). Let $X,Y$ be Banach Spaces, $T:X\to Y$ a continuous linear operator, and $y\in Y$. Solve

$$\left\{\begin{array}{c}min\parallel T\left(x\right)-y\parallel ,\hfill \\ min\parallel x\parallel ,\hfill \end{array}\right.$$

(18)

for $x\in X$.

The infinite-dimensional bounded Tykhonov regularization in the Hilbert space context is described next.

Problem 14.

Let $H,K$ be Hilbert spaces, $T:H\to K$ a continuous linear operator, and $k\in K$. Solve

$$\left\{\begin{array}{c}min\parallel T\left(h\right)-k\parallel ,\hfill \\ min\parallel h\parallel ,\hfill \end{array}\right.$$

(19)

for $h\in H$.

We will find a nontrivial Pareto-optimal solution of Problem 14.

Theorem 6.

Let $H,K$ be Hilbert spaces. Let $T:H\to K$ be a continuous linear operator of closed range. Let $k\in K$. Then, $p_{ker{\left(T\right)}^{\perp }}\left(T^{-1}\left(p_{T\left(H\right)}\left(k\right)\right)\right)$ is a Pareto-optimal solution of (19).

Proof. 

Bear in mind that Theorem 2(3) ensures that $p_{ker{\left(T\right)}^{\perp }}\left(T^{-1}\left(p_{T\left(H\right)}\left(k\right)\right)\right)$ is a singleton; hence, we can take $h_0\in H$ with $\left\{h_0\right\}=p_{ker{\left(T\right)}^{\perp }}\left(T^{-1}\left(p_{T\left(H\right)}\left(k\right)\right)\right)$. In view of Theorem 2(2), $p_{ker{\left(T\right)}^{\perp }}\left(T^{-1}\left(p_{T\left(H\right)}\left(k\right)\right)\right)\subseteq T^{-1}\left(p_{T\left(H\right)}\left(k\right)\right)$. By applying Corollary 3(2), $\arg min\left\{\parallel T\left(h\right)-k\parallel :h\in H\right\}=T^{-1}\left(p_{T\left(H\right)}\left(k\right)\right)$. This implies that $\parallel T\left(h_0\right)-k\parallel \le \parallel T\left(h\right)-k\parallel$ for all $h\in H$. Suppose on the contrary that $h_0$ is not a Pareto-optimal solution of (19). There exists $h_1\in H$ satisfying one of the following two possibilities:

• $\parallel T\left(h_1\right)-k\parallel \le \parallel T\left(h_0\right)-k\parallel$ and $\parallel h_1\parallel <\parallel h_0\parallel$. As a consequence, $\parallel T\left(h_0\right)-k\parallel =\parallel T\left(h_1\right)-k\parallel$, so $h_1\in \arg min\left\{\parallel T\left(h\right)-k\parallel :h\in H\right\}=T^{-1}\left(p_{T\left(H\right)}\left(k\right)\right)$. Finally, by calling again on Theorem 2(2), $\arg min\left\{\parallel h\parallel :h\in T^{-1}\left(p_{T\left(H\right)}\left(k\right)\right)\right\}=p_{ker{\left(T\right)}^{\perp }}$$\left(T^{-1}\left(p_{T\left(H\right)}\left(k\right)\right)\right)$, so $h_0$ is the element of the minimum norm of $T^{-1}\left(p_{T\left(H\right)}\left(k\right)\right)$, reaching the contradiction that $\parallel h_0\parallel \le \parallel h_1\parallel$.
• $\parallel T\left(h_1\right)-k\parallel <\parallel T\left(h_0\right)-k\parallel$ and $\parallel h_1\parallel \le \parallel h_0\parallel$. This is impossible because we have already proven that $\parallel T\left(h_0\right)-k\parallel \le \parallel T\left(h_1\right)-k\parallel$.

As a consequence of the two previous contradictions, we deduce that the singleton $p_{ker{\left(T\right)}^{\perp }}\left(T^{-1}\left(p_{T\left(H\right)}\left(k\right)\right)\right)$ is a Pareto-optimal solution of (19).    □

Under the settings of Theorem 6 and by taking into consideration Lemma 2, the reader should notice that $ker{\left(T^{*}\right)}^{\perp }=T\left(H\right)$, thus $p_{ker{\left(T\right)}^{\perp }}\left(T^{-1}\left(p_{ker{\left(T^{*}\right)}^{\perp }}\left(k\right)\right)\right)=$$p_{ker{\left(T\right)}^{\perp }}\left(T^{-1}\left(p_{T\left(H\right)}\left(k\right)\right)\right)$. Hence, $p_{ker{\left(T\right)}^{\perp }}\left(T^{-1}\left(p_{ker{\left(T^{*}\right)}^{\perp }}\left(k\right)\right)\right)$ is a Pareto-optimal solution of (19). A more intuitive way of understanding Theorem 6 is the following. It is not hard to see, by keeping in mind Theorem 2, that $p_{ker{\left(T\right)}^{\perp }}\left(T^{-1}\left(p_{ker{\left(T^{*}\right)}^{\perp }}\left(k\right)\right)\right)$ is the solution of

$$\left\{\begin{array}{c}min\parallel h\parallel ,\hfill \\ T\left(h\right)=p_{ker{\left(T^{*}\right)}^{\perp }}\left(k\right).\hfill \end{array}\right.$$

(20)

What Corollary 3(2) is saying is that the set of constraints of the above problem,

$$\left\{h\in H:T\left(h\right)=p_{ker{\left(T^{*}\right)}^{\perp }}\left(k\right)\right\},$$

is indeed the set of solutions of

$$\left\{\begin{array}{c}min\parallel T\left(h\right)-k\parallel ,\hfill \\ h\in H.\hfill \end{array}\right.$$

(21)

Corollary 4.

Let $G\in {\mathbb{R}}^{m\times n}$ and $g\in {\mathbb{R}}^m$. Then, $p_{ker{\left(G\right)}^{\perp }}\left(G^{-1}\left(p_{ker{\left(G^t\right)}^{\perp }}\left(g\right)\right)\right)$ is a Pareto-optimal solution of (17).

Proof. 

It only suffices to call on Theorem 6 by taking $H:={\ell }_2^n$, $K:={\ell }_2^m$, $T:=G$, and $k:=g$.    □

A MATLAB encoding for Corollary 4 is available in Appendix C.

4.2. Precise Tykhonov Regularization

The bounded Tykhonov regularization might produce a solution of an excessively small norm. Sometimes, it is precise to obtain a solution of the Tykhonov regularization with a certain predetermined norm. This is what we call the precise Tykhonov regularization.

Problem 15

(Finite-dimensional precise Tykhonov regularization). Let $G\in {\mathbb{R}}^{m\times n}$ with $G\ne 0$ and $ker\left(G\right)\ne \left\{0\right\}$, $g\in {\mathbb{R}}^m$, and $\alpha \ge min\left\{{\parallel \psi \parallel }_2:\psi \in \arg min{\parallel G\psi -g\parallel }_2\right\}$. Solve

$$\left\{\begin{array}{c}{min\parallel G\psi -g\parallel }_2,\hfill \\ {\parallel \psi \parallel }_2=\alpha ,\hfill \end{array}\right.$$

(22)

for $\psi \in {\mathbb{R}}^n$.

Under the settings of Problem 15, by bearing in mind Corollary 3(3), if $ker\left(G\right)=\left\{0\right\}$, then ${\arg min\parallel G\psi -g\parallel }_2$ is a singleton. This is why it is required that $ker\left(G\right)\ne \left\{0\right\}$. The infinite-dimensional version of the precise Tykhonov regularization follows now.

Problem 16

(Infinite-dimensional precise Tykhonov regularization). Let $X,Y$ be Banach Spaces, $T:X\to Y$ a nonzero continuous linear operator with $ker\left(T\right)\ne \left\{0\right\}$, $y\in Y$, and $\alpha \ge min\left\{\parallel x\parallel :x\in \arg min\parallel T\left(x\right)-y\parallel \right\}$. Solve

$$\left\{\begin{array}{c}min\parallel T\left(x\right)-y\parallel ,\hfill \\ \parallel x\parallel =\alpha ,\hfill \end{array}\right.$$

(23)

for $x\in X$.

The infinite-dimensional precise Tykhonov regularization in the Hilbert space context is described next. Observe that, in accordance with Corollary 3(2),

$$min\left\{\parallel h\parallel :h\in \arg min\parallel T\left(h\right)-k\parallel \right\}=\mathrm{dist}\left(0,T^{-1}\left(p_{T\left(H\right)}\left(k\right)\right)\right).$$

Problem 17.

Let $H,K$ be Hilbert spaces, $T:H\to K$ a nonzero continuous linear operator with $ker\left(T\right)\ne \left\{0\right\}$, $k\in K$, and $\alpha \ge \mathrm{dist}\left(0,T^{-1}\left(p_{T\left(H\right)}\left(k\right)\right)\right)$. Solve

$$\left\{\begin{array}{c}min\parallel T\left(h\right)-k\parallel ,\hfill \\ \parallel h\parallel =\alpha ,\hfill \end{array}\right.$$

(24)

for $h\in H$.

Notice that Problem 17 is a single-objective optimization problem. We will find an optimal solution of Problem 17. For this, we will make use of several proper technical results from Banach Space Theory and Operator Theory. Recall that, in a vector space, a linear manifold is a translation of a subspace. The dimension of a linear manifold is by definition the dimension of the subspace.

Theorem 7.

Let X be a Banach Space. Let $M\subseteq X$ be a linear manifold with $dim\left(M\right)\ge 1$. If $\alpha >\mathrm{dist}(0,M)$, then there exists $m_0\in M$ such that $\parallel m_0\parallel =\alpha$.

Proof. 

On the one hand, since $\mathrm{dist}(0,M):=inf\{\parallel m\parallel :m\in M\}$, there exists $m_1\in M$ with $\mathrm{dist}(0,M)\le \parallel m_1\parallel <\alpha$. On the other hand, since $dim\left(M\right)\ge 1$, we have that M is unbounded; thus, there exists $m_2\in M$ with $\parallel m_2\parallel >\alpha$. Consider the continuous function

$$\begin{array}{cccc}\hfill \varphi :& \hfill [0,1]& \to & [0,\infty )\hfill \\ \hfill & \hfill t& \mapsto & \varphi \left(t\right):=\parallel t{m}_2+(1-t)m_1\parallel .\hfill \end{array}$$

(25)

Notice that $\varphi \left(0\right)=\parallel m_1\parallel <\alpha$ and $\varphi \left(1\right)=\parallel m_2\parallel >\alpha$. As a consequence, Bolzano’s Theorem allows the existence of $t_0\in (0,1)$ such that $\varphi \left(t_0\right)=\alpha$. Finally, it only suffices to take $m_0=t_0{m}_2+(1-t_0)m_1$.    □

Observe that Theorem 7 works with the exact same proof if we replace “linear manifold with dimension ≥1” with “unbounded convex subset”. Theorem 7 can actually be accomplished in the Hilbert space setting with much less effort.

Remark 3.

Let $H,K$ be Hilbert spaces. Let $T:H\to K$ be a nonzero continuous linear operator such that $ker\left(T\right)\ne \left\{0\right\}$. Let $k\in K$. Denote $\delta :=\mathrm{dist}\left(0,T^{-1}\left(k\right)\right)=min\left\{\parallel h\parallel :h\in T^{-1}\left(k\right)\right\}$, and let $\alpha \ge \delta$. Fix an arbitrary $h_1\in ker\left(T\right)\backslash \left\{0\right\}$. Notice that $h_2:=\sqrt{{\alpha }^2-{\delta }^2}\frac{h_1}{\parallel h_1\parallel }$ satisfies that $h_2\in ker\left(T\right)$, and by virtue of Theorem 2, for every $h_0\in T^{-1}\left(k\right)$, $p_{ker{\left(T\right)}^{\perp }}\left(h_0\right)+h_2\in T^{-1}\left(k\right)$ and

$$∥p_{ker{\left(T\right)}^{\perp }}\left(h_0\right)+h_2∥=\sqrt{{∥p_{ker{\left(T\right)}^{\perp }}\left(h_0\right)∥}^2+{∥h_2∥}^2}=\sqrt{{\delta }^2+{\alpha }^2-{\delta }^2}=\alpha .$$

Remark 3 allows easily solving the infinite-dimensional precise Tykhonov regularization in the Hilbert space context, that is Problem 17.

Theorem 8.

Let $H,K$ be Hilbert spaces, $T:H\to K$ a nonzero continuous linear operator with $ker\left(T\right)\ne \left\{0\right\}$, $k\in K$, and $\alpha \ge \delta :=\mathrm{dist}\left(0,T^{-1}\left(p_{T\left(H\right)}\left(k\right)\right)\right)$. For every $h_0\in T^{-1}\left(p_{T\left(H\right)}\left(k\right)\right)$ and every $h_1\in ker\left(T\right)\backslash \left\{0\right\}$, an optimal solution of Problem 17 is given by $p_{ker{\left(T\right)}^{\perp }}\left(h_0\right)+\sqrt{{\alpha }^2-{\delta }^2}\frac{h_1}{\parallel h_1\parallel }.$

Proof. 

First off, notice that, according to Remark 3,

$$\begin{array}{ccc}\hfill ∥p_{ker{\left(T\right)}^{\perp }}\left(h_0\right)+\sqrt{{\alpha }^2-{\delta }^2}\frac{h_1}{\parallel h_1\parallel }∥& =& \sqrt{{∥p_{ker{\left(T\right)}^{\perp }}\left(h_0\right)∥}^2+{∥\sqrt{{\alpha }^2-{\delta }^2}\frac{h_1}{\parallel h_1\parallel }∥}^2}\hfill \\ & =& \sqrt{{\delta }^2+{\alpha }^2-{\delta }^2}=\alpha .\hfill \end{array}$$

This means $p_{ker{\left(T\right)}^{\perp }}\left(h_0\right)+\sqrt{{\alpha }^2-{\delta }^2}\frac{h_1}{\parallel h_1\parallel }$ belongs to the feasible region of Problem 17. Next, by applying Corollary 3, $\arg min\left\{\parallel T\left(h\right)-k\parallel :h\in H\right\}=T^{-1}\left(p_{T\left(H\right)}\left(k\right)\right)=h_0+ker\left(T\right)$, because $h_0\in T^{-1}\left(p_{T\left(H\right)}\left(k\right)\right)$. In accordance with Theorem 2, $p_{ker{\left(T\right)}^{\perp }}\left(h_0\right)\in T^{-1}\left(p_{T\left(H\right)}\left(k\right)\right)$. In fact, Theorem 2 ensures that $\arg min\left\{\parallel h\parallel :h\in T^{-1}\left(p_{T\left(H\right)}\left(k\right)\right)\right\}=p_{ker{\left(T\right)}^{\perp }}\left(h_0\right)$. Finally, by using again Corollary 3, if $h\in H$ with $\parallel h\parallel =\alpha$, then

$$\begin{array}{ccc}\hfill ∥T\left(p_{ker{\left(T\right)}^{\perp }}\left(h_0\right)+\sqrt{{\alpha }^2-{\delta }^2}\frac{h_1}{\parallel h_1\parallel }\right)-k∥& =& ∥T\left(h_0\right)-k∥\hfill \\ & =& ∥p_{T\left(H\right)}\left(k\right)-k∥\hfill \\ & =& ∥p_{T{\left(H\right)}^{\perp }}\left(k\right)∥\hfill \\ & \le & \parallel T\left(h\right)-k\parallel .\hfill \end{array}$$

   □

Corollary 5.

Let $G\in {\mathbb{R}}^{m\times n}$ with $G\ne 0$ and $ker\left(G\right)\ne \left\{0\right\}$, $g\in {\mathbb{R}}^m$, and  $\alpha \ge \delta :=min\left\{{\parallel \psi \parallel }_2:\psi \in \arg min{\parallel G\psi -g\parallel }_2\right\}$. For every ${\psi }_0\in G^{-1}\left(p_{ker\left(G^t\right)}\left(g\right)\right)$ and every ${\psi }_1\in ker\left(G\right)\backslash \left\{0\right\}$, $p_{ker{\left(G\right)}^{\perp }}\left({\psi }_0\right)+\sqrt{{\alpha }^2-{\delta }^2}\frac{{\psi }_1}{\parallel {\psi }_1\parallel }$ is an optimal solution of (22).

Proof. 

It only suffices to call on Theorem 8 by taking $H:={\ell }_2^n$, $K:={\ell }_2^m$, $T:=G$, $k:=g$, $h_0:={\psi }_0$, and $h_1:={\psi }_1$.    □

A MATLAB encoding for Corollary 5 is available in Appendix D.

4.3. A Generalization of Theorem 5

As the reader may notice, both Lemma 3 and Theorem 5 are technical results crucial for the development of this manuscript. The following theorem generalizes them.

Theorem 9.

Let X be a Banach Space. Let $P:X\to X$ be a continuous linear projection such that $\parallel I-P\parallel =1$. Let $x_0\in X$. Then, $\mathrm{dist}(x_0,P\left(X\right))=min\{\parallel x_0-y\parallel :y\in P\left(X\right)\}=∥x_0-P\left(x_0\right)∥$. Even more, the previous min is attained at $P\left(x_0\right)$.

Proof. 

It only suffices to observe that, for every $y\in P\left(X\right)$,

$$\parallel x_0-P\left(x_0\right)\parallel =\parallel (I-P)\left(x_0\right)\parallel =\parallel (I-P)(x_0-y)\parallel \le \parallel x_0-y\parallel .$$

   □

5. Conclusions

Let us summarize all the optimization problems we have dealt with throughout this manuscript. The “inclusion” symbol means that the “contained” problem is a particular case of the “continent” problem. The “equal” symbol means that the two involved problems are equivalent in the sense that they have the same set of optimal solutions:

• Problem 1 ⊆ Problem 7⊆ Problem 2.
• Problem 3 = Problem 9⊆ Problem 4.
• Problem 8 ⊆ Problem 10⊆ Problem 4 (assuming $c<d$).
• Problem 8 ⊆ Problem 3 (assuming $c<d$).
• Problem 5 ⊆ Problem 11⊆ Problem 6.
• Problem 12 ⊆ Problem 14⊆ Problem 13.
• Problem 15 ⊆ Problem 17⊆ Problem 16.

We have fully solved Problem 7 (Theorem 2), Problem 8 (Lemma 4), Problem 11 (Corollary 3), Problem 14 (Theorem 6), and Problem 17 (Theorem 8). As a consequence, Problem 1 (Corollary 1), Problem 5, Problem 12 (Corollary 4), and Problem 15 (Corollary 5) are automatically fully solved. With respect to Problem 10, its solution has been approached (Theorem 4).

Appendix A.1. Pseudocode

The pseudocode for Problem 1 consists of the following lines:

• Find ${\psi }_0\in {\mathbb{R}}^n$ such that $G{\psi }_0=g$ (solve $G{\psi }_0=g$).
• Find a basis $K=\{v_1,\cdots ,v_s\}$ of $ker\left(G\right)$.
• Find a basis $O=\{w_1,\cdots ,w_{n-s}\}$ of $ker{\left(G\right)}^{\perp }$.
• Take $B:=K\cup O=\{v_1,\cdots ,v_s,w_1,\cdots ,w_{n-s}\}$ as an ordered basis of ${\mathbb{R}}^n$.
• Find the coordinates $({\alpha }_1,\cdots ,{\alpha }_s,{\alpha }_{s+1},\cdots ,{\alpha }_n)$ of ${\psi }_0$ with respect to B.
• Define $p_{ker{\left(G\right)}^{\perp }}\left({\psi }_0\right)={\alpha }_{s+1}{w}_1+\cdots +{\alpha }_n{w}_{n-s}$.

Appendix A.2. MATLAB Code

We will define a function $\mathtt{MRI}(G,g)$ of two inputs, G and g, which returns one output: $p_{ker{\left(G\right)}^{\perp }}\left({\psi }_0\right)={\alpha }_{s+1}{w}_1+\cdots +{\alpha }_n{w}_{n-s}$.

• function [sol] = MRI(G,g)
•  x_0 = G\g;    % Pseudocode (1)
•  n = length(x_0);
•  K = null(G);   % Pseudocode (2)
•  s = rank(K);
•  O = null(K’);   % Pseudocode (3)
•  B = [K,O];    % Pseudocode (4)
•  X = B\x_0;    % Pseudocode (5)
•  coord = X(s+1:n,1);
•  sol = O*coord;   % Pseudocode (6)
• end

Appendix B.1. Pseudocode

The pseudocode for Problem 1 when $m=1$ consists of the following line:

• Find $\frac{g}{{∥G^t∥}^2}{G}^t$.

Appendix B.2. MATLAB Code

We will define a function $\mathtt{fMRI}(G,g)$ of two inputs, G and g, which returns one output: $\frac{g}{{∥G^t∥}^2}{G}^t$.

•  
• function [sol] = fMRI(G,g)
•  a = norm(G’)
•  b = g/a^2
•  sol = b*G’     % Pseudocode (1)
• end

Appendix C.1. Pseudocode

The pseudocode for Problem 12 consists of the following lines:

• Find a basis $K=\{v_1,\cdots ,v_s\}$ of $ker\left(G^t\right)$.
• Find a basis $O=\{w_1,\cdots ,w_{m-s}\}$ of $ker{\left(G^t\right)}^{\perp }$.
• Take $B:=K\cup O=\{v_1,\cdots ,v_s,w_1,\cdots ,w_{m-s}\}$ as an ordered basis of ${\mathbb{R}}^m$.
• Find the coordinates $({\alpha }_1,\cdots ,{\alpha }_s,{\alpha }_{s+1},\cdots ,{\alpha }_m)$ of g with respect to B.
• Define $p_{ker{\left(G^t\right)}^{\perp }}\left(g\right)={\alpha }_{s+1}{w}_1+\cdots +{\alpha }_m{w}_{m-s}$.
• Apply $\mathtt{MRI}\left(G,p_{ker{\left(G^t\right)}^{\perp }}\left(g\right)\right)$.

Appendix C.2. MATLAB Code

We will define a function $\mathtt{bTR}(G,g)$ of two inputs, G and g, which returns one output: $\mathtt{MRI}\left(G,p_{ker{\left(G^t\right)}^{\perp }}\left(g\right)\right)$.

• function [sol] = bTR(G,g)
•  m = size(G);
•  K = null(G’);   % Pseudocode (1)
•  s = rank(K);
•  O = null(K’);   % Pseudocode (2)
•  B = [K,O];     % Pseudocode (3)
•  X = B\g;     % Pseudocode (4)
•  coord = X(s+1:m,1);
•  p = O*coord;    % Pseudocode (5)
•  sol = MRI(G,p);   % Pseudocode (6)
• end

Appendix D.1. Pseudocode

The pseudocode for Problem 15 consists of the following lines:

• Define ${\psi }_0:=\mathtt{bTR}(G,g)$.
• Define $\delta :=\parallel {\psi }_0{\parallel }_2$.
• Find ${\psi }_1\in {\mathbb{R}}^n\backslash \left\{0\right\}$ such that $G{\psi }_1=0$ (solve $G{\psi }_1=0$).
• Define ${\psi }_0+\sqrt{{\alpha }^2-{\delta }^2}\frac{{\psi }_1}{\parallel {\psi }_1{\parallel }_2}$.

Appendix D.2. MATLAB Code

We will define a function $\mathtt{pTR}(G,g,\alpha )$ of three inputs, G, g, and $\alpha$, which returns one output: ${\psi }_0+\sqrt{{\alpha }^2-{\delta }^2}\frac{{\psi }_1}{\parallel {\psi }_1{\parallel }_2}$.

• function[sol] = pTR(G,g,a)
•  x_0 = bTR(G,g);         % Pseudocode (1)
•  d = norm(x_0);         % Pseudocode (2)
•  X = null(G);
•  x_1 = X(:,1);         % Pseudocode (3)
•  sol = x_0 + sqrt(a^2-d^2)*x_1/norm(x_1); % Pseudocode (4)
• end