A Graduated Non-Convexity Technique for Dealing Large Point Spread Functions

Abstract

This paper focuses on reducing the computational cost of a GNC Algorithm for deblurring images when dealing with full symmetric Toeplitz block matrices composed of Toeplitz blocks. Such a case is widespread in real cases when the PSF has a vast range. The analysis in this paper centers around the class of gamma matrices, which can perform vector multiplications quickly. The paper presents a theoretical and experimental analysis of how $\gamma$-matrices can accurately approximate symmetric Toeplitz matrices. The proposed approach involves adding a minimization step for a new approximation of the energy function to the GNC technique. Specifically, we replace the Toeplitz matrices found in the blocks of the blur operator with $\gamma$-matrices in this approximation. The experimental results demonstrate that the new GNC algorithm proposed in this paper reduces computation time by over $20\%$ compared with its previous version. The image reconstruction quality, however, remains unchanged.

1. Introduction

This paper addresses the problem of reconstructing blurry and noisy images. In particular, we consider blur operators that have a PSF (point spread function) with a vast domain. This case is pervasive; for example, it is common in underwater images (cf. [1,2]). Figure 1 shows the point spread function of the Hubble space telescope camera before NASA corrections. The non-blind problem of restoring images consists of estimating the original image, starting from the observed image and the known blur. This problem is ill-posed in the Hadamard sense (cf. [3]). However, using various regularization techniques, it is possible to turn this problem into a well-posed one that can be solved by minimizing an energy function (cf. [4,5,6]). This function consists of two terms, one that ensures the solution fits the observed data and another that enforces the regularity of the solution.

To produce more realistic restored images, we consider the discontinuities in natural images, particularly around edges where different objects meet (cf. [7]). A possible approach is to use an energy function, which implicitly assumes these discontinuities (cf. [4,5,6,8]). This energy function has a non-convex regularization term. Moreover, to improve the quality of the restored images, it is possible to add constraints that prevent thick boundaries from forming between smooth areas (cf. [4,5]). Thus, the resulting energy function is not convex and cannot be minimized using traditional gradient descent optimization algorithms. Stochastic or deterministic techniques can minimize such a non-convex energy function. By the former methods, it is possible to obtain very accurate results; however, their computational cost is very high (cf. [7]). Deterministic algorithms do not ensure convergence to the ideal solution but allow to obtain adequate reconstruction in lower computational times (cf. [9,10]). The GNC (Graduated Non-Convexity) is one of the most widely used deterministic methods for edge-preserving reconstruction. This technique approximates the energy function using a sequence of approximations that converge to the original one and then solves each approximation using a classical optimization algorithm using the minimum found in the previous approximation as a starting point.

In [8], Blake and Zisserman propose the first GNC algorithm dealing with the denoising problem. Bedini, Gerace, and Tonazzini in [11] present an extension of GNC to restore noisy images, considering the discontinuities’ geometry. Nikolova in [12] proposes a GNC technique to restore noisy blurred images. Boccuto, Gerace, and Pucci in [5] present a GNC algorithm for the deblurring and the denoising imposing the constraint of the line continuation or the non-parallelism constraint alternatively; such an algorithm is referred to as CATILED (Convex Approximation Technique for Interacting Line Elements Deblurring). In this paper, we extend the CATILED technique, in the case of the non-parallelism constraint, to deal with PSFs with a large domain.

The GNC technique has recently been applied to solve many other applications, such as solving the combinatorial data analysis problem of seriation (cf. [13]), stochastic problems (cf. [14]), solving the combinatorial optimization problems defined on the set of partial permutation matrices (cf. [15]), solving the maximum a posteriori inference problem (cf. [16]), pose estimation (cf. [17]), and spatial perception (cf. [18]).

When the blur matrix is full, the computational cost of a GNC algorithm increases considerably; however, this technique remains a good compromise between non-edge-preserving techniques that yield low-accuracy results in a low computational time and stochastic techniques that give qualitatively accurate results with much higher computational cost. Experimental evidence suggests that the most computationally expensive minimization is the first energy function approximation since subsequent minimizations start from a good solution approximation. In this paper, we propose a method to minimize the first convex approximation by approximating each block of the blur operator using matrices that can be efficiently treated using a fast discrete transform. Since each block of the blur operator is a symmetric Toeplitz matrix, we here focus on finding a class of matrices that is easy to handle computationally while providing a good approximation of the Toeplitz matrices. Specifically, we approximate each Toeplitz matrix as the sum of a symmetric circulant and a reverse circulant matrix (cf. [19]). Symmetric circulant matrices have several applications in ordinary and partial differential equations (cf. [20,21,22,23,24]), images and signal restoration (cf. [25,26]), and graph theory (cf. [27,28,29,30,31,32]). Reverse circulant matrices have different applications, such as exponential data fitting and signal processing (cf. [33,34,35,36,37]).

By theoretical and experimental results, we choose a suitable subclass of matrices to use for our approximations. This subclass of matrices is the set of the $\gamma$-matrices presented in [38]. We tested the proposed algorithm in reconstructing artificially blurred images and those affected by natural blurring. These experiments show how using such approximations reduces about a fifth of the CATILED algorithm’s computational costs without affecting the result’s quality. We refer to the technique here proposed as E–CATILED (Extended Convex Approximation Technique for Interacting Line Elements Deblurring).

The paper is structured as follows: in Section 2, we present the problem of image deblurring and the related regularization technique; in Section 3, we recall the CATILED algorithm for the minimization of the energy function; in Section 4, we present the proposed E–CATILED technique; in Section 5, we report our experimental results.

2. Regularization of the Problem

The formulation of the image generation direct problem is

$$\begin{array}{c}\hfill \mathbf{y}=\widehat{A}\mathbf{x}+\mathbf{n},\end{array}$$

where the $n^2$-dimensional vectors $\mathbf{x}$, $\mathbf{y}$ are, respectively, the original and the observed image. We assume that all intensity values are in one column in lexicographic order. The $n^2$-dimensional vector $\mathbf{n}$ expresses the additive noise on the image, which we assume to be independent and identically distributed Gaussian, with zero mean and known variance ${\widehat{\sigma }}^2$.

The $n^2\times n^2$ matrix $\widehat{A}$ represents a translation-invariant blur operation on an image. This operation involves computing a light-intensity weighted average of the neighboring pixels of each pixel in the original image and assigning the result to that pixel in the blurred image. To define the matrix $\widehat{A}$, we use a matrix $M\in {\mathbb{R}}^{(2\widehat{h}+1)\times (2\widehat{h}+1)}$ called blur mask, and we compute the entries of matrix $\widehat{A}$ as

$$\begin{array}{c}{a}_{(i,j),(i+w,j+v)}=\left\{\begin{array}{cc}{m}_{\widehat{h}+1+w,\widehat{h}+1+v},\hfill & \mathrm{if}\left|w\right|,\left|v\right|\le \widehat{h},\hfill \\ & \\ 0,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.\hfill \end{array}$$

Here, in lexicographic notation, the generic index $\left((i,j),(k,l)\right)$ of matrix $\widehat{A}$ is supposed to be equal to $\left((j-1)n+i,(l-1)n+k\right)$. Namely, the blur mask determines the weighting factors used in the weighted averaging operation. Thus, the matrix $\widehat{A}$ becomes a block Toeplitz matrix with Toeplitz blocks (cf. [39]). Note that the size of the blur mask $2\widehat{h}+1$ corresponds to the size of the domain of the PSF (point spread function). If we assume that the blur operator is symmetric in the horizontal and in the vertical direction and the domain of the PSF is vast (that is $2\widehat{h}+1\approx n$), then the full matrix $\widehat{A}$ is symmetric.

The image restoration problem consists of finding an estimation $\mathbf{x}$ of the unknown original image given the blurred image $\mathbf{y}$, the matrix $\widehat{A}$, and the variance of the noise ${\sigma }^2$. This problem is ill-posed in the Hadamard sense; therefore, to solve the problem, some regularization techniques are necessary. Using the second-order difference operators in a regularization technique allows for significantly better results than those obtained by first-order difference operators (cf. [5]). On the other hand, using third-order difference operators yields slightly better results than those obtained with second-order difference operators to the detriment of an excessive increase in computational costs. Therefore, we use second-order difference operators.

A clique c of order 2 is the subset of points of a square grid on which the second-order finite difference is defined. We denote by C the set of all cliques of order 2. More precisely, we consider

$$C=\{c=\{(i,j),(h,l),(r,q)\}:i=h=r,j=l+1=q+2,\mathrm{or}i=h+1=r+2,j=l=q\}.$$

We denote by $D_c\mathbf{x}$ the second-order finite difference operator of the vector $\mathbf{x}$ associated with the clique c, that is, if $c=\left\{\right(i,j),(h,l),(r,q\left)\right\}\in C$, then

$$D_c\mathbf{x}=x_{i,j}-2x_{h,l}+x_{r,q}.$$

Let us introduce the concept of adjacent clique of order 2, used to define the non-parallelism constraint, whose importance is apparent in Figure 2. The blurred image appears in (a); (b) is reconstructed from (a) without imposing the non-parallelism constraint, while the image in (d) is obtained by enforcing it. Although the reconstructions of Figure 2b,d appear similar to the human eye, the underlying quality for the latter is higher, as visible in the corresponding line process plots (c) and (e).

Given a vertical clique

$$c=\left\{\right(i,j),(i+1,j),(i+2,j\left)\right\},i=3,\dots ,n-2,j=1,\dots ,n,$$

we define its preceding clique $c-1$ as follows:

$$\begin{array}{c}\hfill c-1=\left\{\right(i-2,j),(i-1,j),(i,j\left)\right\}.\end{array}$$

If c is a horizontal clique,

$$c=\left\{\right(i,j),(i,j+1),(i,j+2\left)\right\},i=1,\dots ,n,j=3,\dots ,n-2,$$

then its preceding clique $c-1$ is defined by setting

$$\begin{array}{c}\hfill c-1=\left\{\right(i,j-2),(i,j-1),(i,j\left)\right\}.\end{array}$$

A regularized solution $\tilde{\mathbf{x}}$ is defined as a minimizer of the following energy function (cf. [5]).

$$E\left(\mathbf{x}\right)=\parallel \mathbf{y}-\widehat{A}{\mathbf{x}\parallel }^2+\sum _{c\in C}\psi (D_c\left(\mathbf{x}\right),D_{c-1}\left(\mathbf{x}\right)),$$

(1)

where

$$\begin{array}{c}\hfill \psi (t_1,t_2)=\left\{\begin{array}{cc}\overline{g}(t_1,0),\hfill & \mathrm{if}|t_2|<s=\frac{\sqrt{\widehat{\alpha }}}{\widehat{\lambda }},\hfill \\ & \\ \overline{g}(t_1,\widehat{\epsilon }),\hfill & \mathrm{if}|t_2|\ge s,\hfill \end{array}\right.\end{array}$$

(2)

and

$$\begin{array}{c}\hfill \overline{g}(t,k)=\left\{\begin{array}{cc}{\widehat{\lambda }}^2{t}^2,\hfill & \mathrm{if}\left|t\right|<\frac{\sqrt{\widehat{\alpha }+k}}{\widehat{\lambda }},\hfill \\ & \\ \widehat{\alpha }+k,\hfill & \mathrm{if}\left|t\right|\ge \frac{\sqrt{\widehat{\alpha }+k}}{\widehat{\lambda }}.\hfill \end{array}\right.\end{array}$$

The first term of the energy function E in (1) is the so-called data consistency term, while the second additive term in (1) is the smoothness term.

Note that the free parameters in the energy function in (1) are $\widehat{\lambda }$, $\widehat{\alpha }$, and $\widehat{\epsilon }$. The parameter $\widehat{\lambda }$ plays the role of adjusting the degree of smoothness of the solution; $\widehat{\alpha }$ represents the cost to add a discontinuity to the estimated solution, while $\widehat{\epsilon }$ is an extra cost for an adjacent parallel discontinuity. A correct value of these parameters allows for more accurate reconstructions (cf. [40]). Figure 3c presents the function $\psi$ considering $\widehat{\lambda }=1$, $\widehat{\alpha }=80$, and $\widehat{\epsilon }=80$.

3. CATILED Technique

This section presents the CATILED (Convex Approximation Technique for Interacting Line Elements Deblurring) algorithm presented in [5]. Such an algorithm is a GNC (Graduated Non-Convexity) technique (cf. [4,5,8,41,42,43]) that allows minimization of the energy function E given in (1). It is simple to verify that such a function is non-convex. In order to use a gradient descent technique, it is necessary to determine an appropriate initial point near the globular optimum. For this purpose, a GNC technique constructs a family of approximations of the energy function ${\left\{E^{\left(p\right)}\right\}}_p$ such that the first approximation is convex and the last corresponds to the non-convex function. Then, the following algorithm finds an approximation of the global minimum using the family ${\left\{E^{\left(p\right)}\right\}}_p$.

initialize $\mathbf{x}$;

while $E^{\left(p\right)}\ne E$ do

•   find the minimum of the function $E^{\left(p\right)}$ starting from the initial point $\mathbf{x}$;
•   $\mathbf{x}=\arg min{E}^{\left(p\right)}$;
•   update the parameter p.

It is immediate to verify that the first term of the energy function E in (1), called the data consistency term, is convex. Then, finding a first convex approximation of the function E reduces to determining a convex approximation of the second additive term in (1), called the smoothness term.

In [5], the authors first determine a $C^1\left({\mathbb{R}}^2\right)$ family of approximation of the function $\psi$ in (2). Namely, ${\psi }^{\left(0\right)}\equiv \psi$, and for $p\in (0,1]$,

$$\begin{array}{c}\hfill {\psi }^{\left(p\right)}(t_1,t_2)=\left\{\begin{array}{cc}{\overline{g}}^{\left(p\right)}(t_1,0),\hfill & \mathrm{if}\mid t_2\mid \le s,\hfill \\ \\ a^{\left(p\right)}\left(t_1\right){(\mid t_2\mid -s)}^2+{\overline{g}}^{\left(p\right)}(t_1,0),\hfill & \mathrm{if}s<\mid t_2\mid \le {\displaystyle \frac{u\left(p\right)+s}{2}},\hfill \\ \\ -a^{\left(p\right)}\left(t_1\right){(\mid t_2\mid -u\left(p\right))}^2+{\overline{g}}^{\left(p\right)}(t_1,\widehat{\epsilon }),\hfill & \mathrm{if}{\displaystyle \frac{u\left(p\right)+s}{2}}<\mid t_2\mid <u\left(p\right),\hfill \\ \\ {\overline{g}}^{\left(p\right)}(t_1,\widehat{\epsilon }),\hfill & \mathrm{otherwise},\hfill \end{array}\right.\end{array}$$

(3)

where $u\left(p\right)=s+pz$, with an arbitrary $z>0$.

The function ${\overline{g}}^{\left(p\right)}(t,k)$ is

$${\overline{g}}^{\left(p\right)}(t,k)=\left\{\begin{array}{cc}{\widehat{\lambda }}^2{t}^2\hfill & \mathrm{if}\left|t\right|<q_p\left(k\right),\hfill \\ & \\ \widehat{\alpha }+k-{\displaystyle \frac{{\tau }^{\left(p\right)}}{2}}\left(\right|t|-r_p{\left(k\right))}^2\hfill & \mathrm{if}{q}_p\left(k\right)\le \left|t\right|\le r_p\left(k\right),\hfill \\ & \\ \widehat{\alpha }+k\hfill & \mathrm{if}\left|t\right|>r_p\left(k\right),\hfill \end{array}\right.$$

where

$$q_p\left(k\right)=\frac{\sqrt{\widehat{\alpha }+k}}{{\widehat{\lambda }}^2}{\left(\frac{2}{{\tau }^{\left(p\right)}}+\frac{1}{{\widehat{\lambda }}^2}\right)}^{-1/2},$$

with ${\tau }^{\left(p\right)}={\tau }^{*}/p$, where ${\tau }^{*}>0$ is an arbitrary constant, and

$$r_p\left(k\right)=\frac{\widehat{\alpha }+k}{{\widehat{\lambda }}^2{q}_p\left(k\right)}.$$

The function $a^{\left(p\right)}\left(t\right)$ is

$$a^{\left(p\right)}\left(t\right)=2\frac{{\overline{g}}^{\left(p\right)}(t,\widehat{\epsilon })-{\overline{g}}^{\left(p\right)}(t,0)}{{[u\left(p\right)-s]}^2}.$$

Thus, the first convex approximation of $\psi$ of class $C^1\left({\mathbb{R}}^2\right)$ in the CATILED technique is

$$\begin{array}{c}\hfill {\psi }^{\left(2\right)}(t_1,t_2)==\left\{\begin{array}{cc}{\widehat{\lambda }}^2{t}_1^2,\hfill & \mathrm{if}\mid t_1\mid <q_1\left(0\right),\hfill \\ \\ 2{\widehat{\lambda }}^2{q}_1\left(0\right)\left|t_1\right|-{\widehat{\lambda }}^2{q}_1^2\left(0\right),\hfill & \mathrm{if}\mid t_1\mid \ge q_1\left(0\right).\hfill \end{array}\right.\end{array}$$

(4)

For $p\in [1,2]$,

$$\begin{array}{c}\hfill {\psi }^{\left(p\right)}=(p-1){\psi }^{\left(2\right)}+(2-p){\psi }^{\left(1\right)},\end{array}$$

where ${\psi }^{\left(1\right)}$ and ${\psi }^{\left(2\right)}$ are given in (3) and (4) respectively. In the CATILED algorithm, the parameter p varies from 2 to 0 with a fixed step h.

Note that all variables used in this section are necessary to make all approximations ${\psi }^{\left(p\right)}$ of the function $\psi$, for $p\in (0,2]$, of class $C^1\left({\mathbb{R}}^2\right)$ (see [5] for details). In order to obtain a graphical view, Figure 3a–c show the graphs of the functions ${\psi }^{\left(2\right)}$, ${\psi }^{\left(1\right)}$, and ${\psi }^{\left(0\right)}\equiv \psi$ when $\widehat{\lambda }=1$, $\widehat{\alpha }=80$ and $\widehat{\epsilon }=80$.

The different approximations are minimized by the NL-SOR (Non-Linear Successive Over Relation) algorithm (cf. [5,8]). In this algorithm, in each iteration, the solution is updated along the opposite direction of the gradient of the energy function. Thus, the current solution should be multiplied by ${\widehat{A}}^T\widehat{A}$ in the computation of the data consistency term component of the gradient. Since $\widehat{A}$ is a full matrix, this operation is extremely costly.

4. E–CATILED Technique

It is possible to verify experimentally that the more expensive minimization is the first since the others start from a good solution approximation. Hence, in this paper, when we minimize the first convex approximation, we propose to approximate every block of the operator $\widehat{A}$ through matrices whose products can be computed by a suitable fast discrete transform.

Since every block of $\widehat{A}$ is a symmetric Toeplitz matrix, we now deal with determining a class of matrices that is easy to handle from the computational point of view that provide a good approximation of the Toeplitz matrices. In particular, in this paper, we approximate each Toeplitz matrix by the sum between a symmetric circulant and a reverse circulant matrix.

4.1. Spectral Characterization of $\beta$-Matrices

Given $A\in {\mathbb{C}}^{n\times n}$, we below denote by $A^{*}$ the transpose conjugate of A. In this and the following subsection, for simplicity of notation, we consider the indices of the $n\times n$-matrices and n-vectors to vary between 0 and $n-1$. We begin with presenting a class of simultaneously diagonalizable matrices recently proposed in [38]. Let n be a fixed positive integer, and $Q_n={\left(q_{k,j}\right)}_{k,j}\in {\mathbb{R}}^{n\times n}$, where

$$\begin{array}{c}\hfill q_{k,j}=\left\{\begin{array}{cc}{\alpha }_{j}cos\left({\displaystyle \frac{2\pi kj}{n}}\right)\hfill & \mathrm{if}0\le j\le \lfloor n/2\rfloor ,\hfill \\ \\ {\alpha }_{j}sin\left({\displaystyle \frac{2\pi k(n-j)}{n}}\right)\hfill & \mathrm{if}\lfloor n/2\rfloor \le j\le n-1,\hfill \end{array}\right.\end{array}$$

(5)

and

$$\begin{array}{c}\hfill {\alpha }_j=\left\{\begin{array}{cc}{\displaystyle \frac{1}{\sqrt{n}}}=\overline{\alpha }\hfill & \mathrm{if}j=0,\mathrm{or}j=n/2\mathrm{if}n\mathrm{is}\mathrm{even},\hfill \\ \\ \sqrt{{\displaystyle \frac{2}{n}}}=\tilde{\alpha }\hfill & \mathrm{otherwise}.\hfill \end{array}\right.\end{array}$$

Set

$$\begin{array}{c}\hfill Q_n=\left({\mathbf{q}}^{\left(0\right)}\right|{\mathbf{q}}^{\left(1\right)}\left|\cdots \right|{\mathbf{q}}^{\left(\lfloor \frac{n}{2}\rfloor \right)}\left|{\mathbf{q}}^{\left(\lfloor \frac{n+1}{2}\rfloor \right)}\right|\cdots \left|{\mathbf{q}}^{(n-2)}\right|{\mathbf{q}}^{(n-1)}),\end{array}$$

with

$$\begin{array}{c}\hfill {\mathbf{q}}^{\left(0\right)}={\displaystyle \frac{1}{\sqrt{n}}}{\left(11\cdots 1\right)}^T={\displaystyle \frac{1}{\sqrt{n}}}{\mathbf{u}}^{\left(0\right)},\end{array}$$

(6)

$$\begin{array}{ccc}\hfill {\mathbf{q}}^{\left(j\right)}& =& \sqrt{{\displaystyle \frac{2}{n}}}{\left(1cos\left({\displaystyle \frac{2\pi j}{n}}\right)\cdots cos\left({\displaystyle \frac{2\pi j(n-1)}{n}}\right)\right)}^T=\sqrt{{\displaystyle \frac{2}{n}}}{\mathbf{u}}^{\left(j\right)},\hfill \\ \hfill {\mathbf{q}}^{(n-j)}& =& \sqrt{{\displaystyle \frac{2}{n}}}{\left(0sin\left({\displaystyle \frac{2\pi j}{n}}\right)\cdots sin\left({\displaystyle \frac{2\pi j(n-1)}{n}}\right)\right)}^T=\sqrt{{\displaystyle \frac{2}{n}}}{\mathbf{v}}^{\left(j\right)},\hfill \end{array}$$

(7)

$j=1$, $2,\dots ,\lfloor \frac{n-1}{2}\rfloor$. If n is even, we have

$$\begin{array}{c}\hfill {\mathbf{q}}^{(n/2)}={\displaystyle \frac{1}{\sqrt{n}}}{\left(1-11-1\cdots -1\right)}^T={\displaystyle \frac{1}{\sqrt{n}}}{\mathbf{u}}^{(n/2)}.\end{array}$$

(8)

Note that $Q_n$ is an orthonormal matrix (cf. [44]).

Given a vector $\mathbf{\lambda }\in {\mathbb{C}}^n$, $\mathbf{\lambda }={({\lambda }_0{\lambda }_1\cdots {\lambda }_{n-1})}^T$, we define

$$\begin{array}{c}\hfill \mathrm{diag}\left(\mathbf{\lambda }\right)=\mathsf{\Lambda }=\left(\begin{array}{cccccc}{\lambda }_0& 0& 0& \dots & 0& 0\\ 0& {\lambda }_1& 0& \dots & 0& 0\\ 0& 0& {\lambda }_2& \dots & 0& 0\\ ⋮& ⋮& ⋮& \ddots & ⋮& ⋮\\ 0& 0& 0& \dots & {\lambda }_{n-2}& 0\\ 0& 0& 0& \dots & 0& {\lambda }_{n-1}\end{array}\right)\in {\mathbb{C}}^{n\times n}.\end{array}$$

We recall that a vector $\mathbf{\lambda }\in {\mathbb{R}}^n$, $\mathbf{\lambda }={({\lambda }_0{\lambda }_1\cdots {\lambda }_{n-1})}^T$ is said to be symmetric if ${\lambda }_j={\lambda }_{n-j}$ for $j=0$, $1,\dots ,\lfloor n/2\rfloor$; otherwise, $\mathbf{\lambda }$ is said to be asymmetric if ${\lambda }_j=-{\lambda }_{n-j}$ for $j=0$, $1,\dots ,\lfloor n/2\rfloor$.

Let ${\mathcal{G}}_n$ be the space of the following simultaneously diagonalizable matrices

$$\begin{array}{c}\hfill {\mathcal{G}}_n=\mathrm{sd}\left(Q_n\right)=\{Q_n\mathsf{\Lambda }{Q}_n^T:\mathsf{\Lambda }=\mathrm{diag}\left(\mathbf{\lambda }\right),\mathbf{\lambda }\in {\mathbb{R}}^n\}.\end{array}$$

The elements of such a class are called γ–matrices. Moreover, in [38], the following classes are presented

$$\begin{array}{c}\hfill {\mathcal{C}}_n=\{Q_n\mathsf{\Lambda }{Q}_n^T:\mathsf{\Lambda }=\mathrm{diag}\left(\mathbf{\lambda }\right),\mathbf{\lambda }\in {\mathbb{R}}^n,\mathbf{\lambda }\mathrm{is}\mathrm{symmetric}\},\end{array}$$

$$\begin{array}{c}\hfill {\mathcal{B}}_n=\{Q_n\mathsf{\Lambda }{Q}_n^T:\mathsf{\Lambda }=\mathrm{diag}\left(\mathbf{\lambda }\right),\mathbf{\lambda }\in {\mathbb{R}}^n,\mathbf{\lambda }\mathrm{is}\mathrm{asymmetric}\},\end{array}$$

It is possible to see that ${\mathcal{G}}_n$ is a matrix algebra of dimension n, ${\mathcal{C}}_n$ is a subalgebra of ${\mathcal{G}}_n$ of dimension $\lfloor \frac{n}{2}\rfloor +1$, and ${\mathcal{B}}_n$ is a linear subspace of ${\mathcal{G}}_n$ of dimension $\lfloor \frac{n-1}{2}\rfloor$ (see [45]). The following results hold.

Proposition 1

([45]). One has

$$\begin{array}{c}\hfill {\mathcal{G}}_n={\mathcal{C}}_n\oplus {\mathcal{B}}_n,\end{array}$$

where ⊕ is the orthogonal sum and $\langle ·,·\rangle$ denotes the Frobenius product, defined by

$$\langle G_1,G_2\rangle =\mathrm{tr}\left(G_1^T{G}_2\right),G_1,G_2\in {\mathcal{G}}_n,$$

where $tr\left(G\right)$ is the trace of the matrix G.

We recall the definition of the classical Hartley matrix (see also [19] and the references therein). If n is odd, we have

$$\begin{array}{c}\hfill H_n={\displaystyle \frac{1}{\sqrt{n}}}\left({\mathbf{u}}^{\left(0\right)}{\mathbf{u}}^{\left(1\right)}+{\mathbf{v}}^{\left(1\right)}\dots {\mathbf{u}}^{\left(\frac{n-1}{2}\right)}+{\mathbf{v}}^{\left(\frac{n-1}{2}\right)}{\mathbf{u}}^{\left(\frac{n-1}{2}\right)}-{\mathbf{v}}^{\left(\frac{n-1}{2}\right)}\dots {\mathbf{u}}^{\left(1\right)}-{\mathbf{v}}^{\left(1\right)}\right).\end{array}$$

When n is even, we obtain

$$\begin{array}{c}\hfill H_n={\displaystyle \frac{1}{\sqrt{n}}}\left({\mathbf{u}}^{\left(0\right)}{\mathbf{u}}^{\left(1\right)}+{\mathbf{v}}^{\left(1\right)}\dots {\mathbf{u}}^{\left(\frac{n}{2}-1\right)}+{\mathbf{v}}^{\left(\frac{n}{2}-1\right)}{\mathbf{u}}^{\left(\frac{n}{2}\right)}{\mathbf{u}}^{\left(\frac{n}{2}-1\right)}-{\mathbf{v}}^{\left(\frac{n}{2}-1\right)}\dots {\mathbf{u}}^{\left(1\right)}-{\mathbf{v}}^{\left(1\right)}\right).\end{array}$$

It is not difficult to see that

$$\begin{array}{c}\hfill H_n=Q_n{Y}_n,\end{array}$$

where $Y_n={\left(y_{k,j}\right)}_{k,j}\in {\mathbb{R}}^{n\times n}$, is a diagonal matrix with

$$\begin{array}{c}\hfill y_{k,j}=\left\{\begin{array}{cc}1\hfill & \mathrm{if}k=j=0,\hfill \\ \frac{1}{\sqrt{2}}\hfill & \mathrm{if}k=j\mathrm{and}1\le k\le \frac{n-1}{2},\hfill \\ \frac{1}{\sqrt{2}}\hfill & \mathrm{if}k+j=n\mathrm{and}1\le k\le n-1,\hfill \\ -\frac{1}{\sqrt{2}}\hfill & \mathrm{if}k=j\mathrm{and}\frac{n+1}{2}\le k\le n-1,\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array}\right.\end{array}$$

if n is odd, and

$$\begin{array}{c}\hfill y_{k,j}=\left\{\begin{array}{cc}1\hfill & \mathrm{if}k=j=0\mathrm{or}k=j=\frac{n}{2},\hfill \\ \frac{1}{\sqrt{2}}\hfill & \mathrm{if}k=j\mathrm{and}1\le k\le \frac{n}{2}-1,\hfill \\ \frac{1}{\sqrt{2}}\hfill & \mathrm{if}k+j=n\mathrm{and}1\le k\le n-1,\hfill \\ -\frac{1}{\sqrt{2}}\hfill & \mathrm{if}k=j\mathrm{and}\frac{n}{2}+1\le k\le n-1,\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array}\right.\end{array}$$

if n is even. Now, set

$$\begin{array}{c}\hfill {\mathcal{H}}_n=\mathrm{sd}\left(H_n\right)=\{H_n\mathsf{\Lambda }{H}_n^T:\mathsf{\Lambda }=\mathrm{diag}\left(\mathbf{\lambda }\right),\mathbf{\lambda }\in {\mathbb{R}}^n\}.\end{array}$$

(9)

It is not difficult to see that

$$\begin{array}{ccc}\hfill {\mathcal{C}}_n& =& \{Q_n\mathsf{\Lambda }{Q}_n^T:\mathsf{\Lambda }=\mathrm{diag}\left(\mathbf{\lambda }\right),\mathbf{\lambda }\in {\mathbb{R}}^n,\mathbf{\lambda }\mathrm{is}\mathrm{symmetric}\}0\hfill \\ & =& \{H_n\mathsf{\Lambda }{H}_n^T:\mathsf{\Lambda }=\mathrm{diag}\left(\mathbf{\lambda }\right),\mathbf{\lambda }\in {\mathbb{R}}^n,\mathbf{\lambda }\mathrm{is}\mathrm{symmetric}\}.\hfill \end{array}$$

(10)

From (9) and (10), it follows that

$$\begin{array}{c}\hfill {\mathcal{H}}_n={\mathcal{C}}_n\oplus {\mathcal{F}}_n,\end{array}$$

where

$$\begin{array}{c}\hfill {\mathcal{F}}_n=\{H_n\mathsf{\Lambda }{H}_n^T:\mathsf{\Lambda }=\mathrm{diag}\left(\mathbf{\lambda }\right),\mathbf{\lambda }\in {\mathbb{R}}^n,\mathbf{\lambda }\mathrm{is}\mathrm{asymmetric}\}.\end{array}$$

The Fourier matrix is defined by $F_n={\left(f_{k,l}\right)}_{k,l}\in {\mathbb{C}}^{n\times n}$, where

$$\begin{array}{c}\hfill f_{k,l}={\displaystyle \frac{1}{\sqrt{n}}}{\omega }_n^{kl},k,l=0,1,\dots ,n-1,\end{array}$$

with ${\omega }_n=e^{\frac{2\pi \mathrm{i}}{n}}$. Let ${\mathcal{W}}_n$ be the space of all real matrices simultaneously diagonalizable by $F_n$, that is,

$$\begin{array}{c}\hfill {\mathcal{W}}_n=\mathrm{sd}\left(F_n\right)=\{F_n\mathsf{\Lambda }{F}_n^{*}\in {\mathbb{R}}^{n\times n}:\mathsf{\Lambda }=\mathrm{diag}\left(\mathbf{\lambda }\right),\mathbf{\lambda }\in {\mathbb{C}}^n\}.\end{array}$$

It is not difficult to see that ${\mathcal{W}}_n$ is a commutative matrix algebra. Moreover, we define the following class:

$$\begin{array}{c}\hfill {\mathcal{A}}_n=\{F_n\mathsf{\Lambda }{F}_n^{*}:\mathsf{\Lambda }=\mathrm{diag}\left(\mathbf{\lambda }\right),\mathbf{\lambda }\in {\left(\mathrm{i}\mathbb{R}\right)}^n,\mathbf{\lambda }\mathrm{is}\mathrm{asymmetric}\}.\end{array}$$

Finally, we define the β-matrices as the matrices belonging to the following set:

$$\begin{array}{c}\hfill {\mathcal{V}}_n={\mathcal{C}}_n\oplus {\mathcal{B}}_n\oplus {\mathcal{F}}_n\oplus {\mathcal{A}}_n.\end{array}$$

4.2. Structural Characterizations of $\beta$-Matrices

In this subsection, we show that ${\mathcal{V}}_n$ coincides with the direct sum of the sets of all real circulant matrices and of all reverse circulant matrices.

We consider the set of families

$$\begin{array}{ccc}\hfill {\mathcal{L}}_{n,k}& =& \{A\in {\mathbb{R}}^{n\times n}:\mathrm{there}\mathrm{is}\mathbf{a}={(a_0\dots a_{n-1})}^T\in {\mathbb{R}}^n\mathrm{with}{a}_{l,j}=a_{(j+kl)modn}\},\hfill \\ \hfill {\mathcal{K}}_{n,k}& =& \{A\in {\mathbb{R}}^{n\times n}:\mathrm{there}\mathrm{is}\mathrm{a}\mathrm{symmetric}\mathbf{a}={(a_0\dots a_{n-1})}^T\in {\mathbb{R}}^n\hfill \\ & & \mathrm{with}{a}_{l,j}=a_{(j+kl)modn}\},\hfill \\ \hfill {\mathcal{J}}_{n,k}& =& \{A\in {\mathbb{R}}^{n\times n}:\mathrm{there}\mathrm{is}\mathrm{a}\mathrm{symmetric}\mathbf{a}={(a_0\dots a_{n-1})}^T\in {\mathbb{R}}^n\hfill \\ & & \mathrm{with}\sum _{t=0}^{n-1}{a}_t=0,\sum _{t=0}^{n-1}{(-1)}^t{a}_t=0\mathrm{when}n\mathrm{is}\mathrm{even},\mathrm{and}{a}_{l,j}=a_{(j+kl)modn}\},\hfill \end{array}$$

where $k\in \{1,2,\dots ,n-1\}$.

When $k=n-1$, ${\mathcal{L}}_{n,n-1}$ is the class of all real circulant matrices, that is, the family of those matrices $C\in {\mathbb{R}}^{n\times n}$ such that every row, after the first, has the elements of the previous one shifted cyclically one place right (see, e.g., [46]).

Given a vector $\mathbf{c}\in {\mathbb{R}}^n$, $\mathbf{c}={(c_0{c}_1\cdots c_{n-1})}^T$, let us define

$$\begin{array}{c}\hfill \mathrm{circ}\left(\mathbf{c}\right)=C=\left(\begin{array}{cccccc}{c}_0& c_1& c_2& \dots & c_{n-2}& c_{n-1}\\ c_{n-1}& c_0& c_1& \dots & c_{n-3}& c_{n-2}\\ c_{n-2}& c_{n-1}& c_0& \ddots & c_{n-4}& c_{n-3}\\ ⋮& ⋮& \ddots & \ddots & \ddots & ⋮\\ c_2& c_3& c_4& \ddots & c_0& c_1\\ c_1& c_2& c_3& \dots & c_{n-1}& c_0\end{array}\right),\end{array}$$

where $C\in {\mathcal{L}}_{n,n-1}$.

Theorem 1

(Theorems 3.2.2 and 3.2.3 in [46]). The following result holds:

$$\begin{array}{c}\hfill {\mathcal{W}}_n={\mathcal{L}}_{n,n-1}.\end{array}$$

As a consequence of this theorem, we obtain that the n eigenvectors of every circulant matrix $C\in {\mathbb{R}}^{n\times n}$ are given by

$$\begin{array}{c}\hfill {\mathbf{w}}^{\left(j\right)}={(1{\omega }_n^j{\omega }_n^{2j}\cdots {\omega }_n^{(n-1)j})}^T,\end{array}$$

and the eigenvalues of a matrix $C=$circ$\left(\mathbf{c}\right)\in {\mathcal{F}}_n$ are expressed by

$${\displaystyle {\lambda }_j={\mathbf{c}}^T{\mathbf{w}}^{\left(j\right)}=\sum _{k=0}^{n-1}{c}_k{\omega }_n^{jk},j=0,1,\dots ,n-1.}$$

Now, we present some results about symmetric circulant real matrices. Observe that if $C=$ circ$\left(\mathbf{c}\right)$, with $\mathbf{c}\in {\mathbb{R}}^n$, then C is symmetric if and only if $\mathbf{c}$ is symmetric. Thus, the class of all real symmetric circulant matrices coincides with ${\mathcal{K}}_{n,n-1}$ and has dimension $\lfloor \frac{n}{2}\rfloor +1$ over $\mathbb{R}$.

Theorem 2

(see, e.g., (§4 in [27]), (Lemma 3 in [44])). Let $C\in {\mathcal{K}}_{n,n-1}$. Then, the set of all eigenvectors of C can be expressed as $\{{\mathbf{q}}^{\left(0\right)}$, ${\mathbf{q}}^{\left(1\right)}$, …, ${\mathbf{q}}^{(n-1)}\}$, where ${\mathbf{q}}^{\left(j\right)}$, $j=0$, $1,\dots ,n-1$, is as in(6)–(8).

Note that from Theorem 2 it follows that the set of all real symmetric circulant matrices is contained in ${\mathcal{G}}_n$. The next result holds.

Theorem 3.

(see, e.g., (§1.2 in [47]), (§4 in [27]), (Theorem 1 in [48])). Let $C=\mathrm{circ}\left(\mathbf{c}\right)\in {\mathcal{K}}_{n,n-1}$. Then, the eigenvalues ${\lambda }_j$ of C are given by

$$\begin{array}{c}\hfill {\lambda }_j={\mathbf{c}}^T{\mathbf{u}}^{\left(j\right)},j=0,1,\dots ,\lfloor \frac{n}{2}\rfloor ,\end{array}$$

(11)

where the ${\mathbf{u}}^{\left(j\right)}$’s are as in (7). Moreover, for $j=1$, $2,\dots ,\lfloor \frac{n-1}{2}\rfloor$ it is

$$\begin{array}{c}\hfill {\lambda }_j={\lambda }_{n-j}.\end{array}$$

From Theorem 3, it follows that, if C is a real symmetric circulant matrix and ${\mathbf{\lambda }}^{\left(C\right)}$ is the set of its eigenvalues, then ${\mathbf{\lambda }}^{\left(C\right)}$ is symmetric, thanks to (11). Hence, ${\mathcal{K}}_{n,n-1}\subset {\mathcal{C}}_n.$ Thus, ${\mathcal{C}}_n$ coincides with the class of symmetric circulant matrices ${\mathcal{K}}_{n,n-1}$ since these two vector spaces have the same dimension.

If $k=1$, then ${\mathcal{L}}_{n,1}$ is the set of all real reverse circulant (or real anti-circulant) matrices, which is the class of all matrices $B\in {\mathbb{R}}^{n\times n}$ such that every row, after the first, has the elements of the previous one shifted cyclically one place left (see, e.g., [46]). Given a vector $\mathbf{b}={(b_0{b}_1\cdots b_{n-1})}^T\in {\mathbb{R}}^n$, set

$$\begin{array}{c}\hfill \mathrm{rcirc}\left(\mathbf{b}\right)=B=\left(\begin{array}{cccccc}{b}_0& b_1& b_2& \dots & b_{n-2}& b_{n-1}\\ b_1& b_2& b_3& \dots & b_{n-1}& b_0\\ b_2& b_3& b_4& \dots & b_0& b_1\\ ⋮& ⋮& ⋮& \dots & ⋮& ⋮\\ b_{n-2}& b_{n-1}& b_0& \dots & b_{n-4}& b_{n-3}\\ b_{n-1}& b_0& b_1& \dots & b_{n-3}& b_{n-2}\end{array}\right),\end{array}$$

with $B\in {\mathcal{L}}_{n,1}$.

Observe that every matrix $B\in {\mathcal{B}}_{n,1}$ is symmetric, and the set ${\mathcal{L}}_{n,1}$ is a linear space over $\mathbb{R}$, but not an algebra. Note that, if $B_1$, $B_2\in {\mathcal{L}}_{n,1}$, then $B_1{B}_2$, $B_2{B}_1\in {\mathcal{L}}_{n,n-1}$ (see Theorem 5.1.2 in [46]). In Appendix A, we prove that

$${\mathcal{B}}_n={\mathcal{J}}_{n,1}.$$

Proposition 2.

Let $B=$ rcirc$\left(\mathbf{b}\right)\in {\mathcal{B}}_n$. Then, the eigenvalues ${\lambda }_j^{\left(B\right)}$ of B, can be expressed as

$$\begin{array}{c}\hfill {\lambda }_j^{\left(B\right)}={\mathbf{b}}^T{\mathbf{u}}^{\left(j\right)},j=0,1,\dots ,\lfloor \frac{n}{2}\rfloor ,\end{array}$$

(12)

where the ${\mathbf{u}}^{\left(j\right)}$’s are as in(7). Moreover, for $j=1$, $2,\dots \lfloor \frac{n-1}{2}\rfloor$, we obtain

$${\lambda }_{n-j}^{\left(B\right)}=-{\lambda }_j^{\left(B\right)}.$$

Furthermore, it is ${\lambda }_0^{\left(B\right)}=0$, and ${\lambda }_{n/2}^{\left(B\right)}=0$ if n is even.

Proof. 

See [45]. □

We note that

$$\begin{array}{c}\hfill {\mathcal{F}}_n=\{A\in {\mathcal{L}}_{n,1}:\mathrm{there}\mathrm{is}\mathrm{an}\mathrm{asymmetric}\mathbf{a}\in {\mathbb{R}}^n\mathrm{with}A=\mathrm{rcirc}\left(\mathbf{a}\right)\}\end{array}$$

(see also [19]).

Proposition 3.

One has

$$\begin{array}{c}\hfill {\mathcal{A}}_n=\{A\in {\mathcal{L}}_{n,n-1}:\mathit{there}\mathit{is}\mathit{an}\mathit{asymmetric}\mathbf{a}\in {\mathbb{R}}^n\mathit{with}A=\mathrm{circ}\left(\mathbf{a}\right)\}.\end{array}$$

Proof. 

See [49]. □

From Proposition 3, it follows that

$$\begin{array}{c}\hfill {\mathcal{L}}_{n,n-1}={\mathcal{C}}_n\oplus {\mathcal{A}}_n.\end{array}$$

Hence, we obtain

$$\begin{array}{c}\hfill {\mathcal{V}}_n={\mathcal{C}}_n\oplus {\mathcal{B}}_n\oplus {\mathcal{F}}_n\oplus {\mathcal{A}}_n={\mathcal{L}}_{n,1}\oplus {\mathcal{L}}_{n,n-1}.\end{array}$$

At each iteration of NL–SOR, we have to multiply a vector by ${\widehat{A}}^T\widehat{A}$, where $\widehat{A}$ is the blur matrix. Since $\widehat{A}$ is a Toeplitz block matrix with Toeplitz blocks, each block of the ${\widehat{A}}^T\widehat{A}$ matrix is composed of symmetric Toeplitz matrices added and multiplied together. Since we approximate every symmetric Toeplitz matrix with a matrix belonging to ${\mathcal{V}}_n$, we now observe that ${\mathcal{V}}_n$ is closed under the operations of addition and multiplication. Indeed, it is not difficult to see that ${\mathcal{V}}_n$ is closed under the operation of sum between matrices. Moreover, ${\mathcal{L}}_{n,1}$ is closed under the operation of multiplication, and if $V_1,V_2\in {\mathcal{L}}_{n,1}$, then $V_1{V}_2\in {\mathcal{L}}_{n,n-1}$, if $V_1\in {\mathcal{L}}_{n,n-1}$ and $V_2\in {\mathcal{L}}_{n,1}$, then $V_1{V}_2\in {\mathcal{L}}_{n,1}$ (see, e.g., [46]).

4.3. Inversion of $\beta$-Matrices

We now analyze the conditions under which a $\beta$-matrix admits inverse.

Proposition 4.

The eigenvalues ${\lambda }_j^{\left(F\right)}$ of $F=$ rcirc$\left(\mathbf{f}\right)\in {\mathcal{F}}_n$, are given by

$$\begin{array}{c}\hfill {\lambda }_j^{\left(F\right)}={\mathbf{f}}^T{\mathbf{v}}^{\left(j\right)},j=0,1,\dots ,\lfloor \frac{n}{2}\rfloor ,\end{array}$$

where the ${\mathbf{v}}^{\left(j\right)}$’s are as in(7). Moreover, for $j=1$, $2,\dots \lfloor \frac{n-1}{2}\rfloor$, we obtain

$${\lambda }_{n-j}^{\left(F\right)}=-{\lambda }_j^{\left(F\right)}.$$

Proof. 

See [45]. □

Proposition 5.

The eigenvalues ${\lambda }_j^{\left(A\right)}$ of $A=$ circ$\left(\mathbf{a}\right)\in {\mathcal{A}}_n$, are given by

$$\begin{array}{c}\hfill {\lambda }_j^{\left(A\right)}=\mathrm{i}{\mathbf{a}}^T{\mathbf{v}}^{\left(j\right)},j=0,1,\dots ,\lfloor \frac{n}{2}\rfloor ,\end{array}$$

where the ${\mathbf{v}}^{\left(j\right)}$’s are as in(7), and for $j=1$, $2,\dots \lfloor \frac{n-1}{2}\rfloor$, we obtain

$${\lambda }_{n-j}^{\left(A\right)}=-{\lambda }_j^{\left(A\right)}.$$

Proof. 

See [45]. □

It is not difficult to see that, given $C\in {\mathcal{C}}_n$ and $V\in {\mathcal{V}}_n$, the eigenvalues of $CV$ are equal to those of $VC$ and are given by

$${\lambda }_j^{\left(CV\right)}={\lambda }_j^{\left(VC\right)}={\lambda }_j^{\left(C\right)}{\lambda }_j^{\left(V\right)},j=0,1,\dots ,n-1.$$

Now, we present the next lemma.

Lemma 1.

The following properties hold.

(i) 

Let $B\in {\mathcal{B}}_n$, $B=$ rcirc$\left(\mathbf{b}\right)$, and $F\in {\mathcal{F}}_n$, $F=$ rcirc$\left(\mathbf{f}\right)$. Then, $BF\in {\mathcal{A}}_n$, and the eigenvalues of $BF$ are expressed by

$$\begin{array}{c}\hfill \begin{array}{cc}{\lambda }_j^{\left(BF\right)}=\mathrm{i}{\lambda }_j^{\left(B\right)}{\lambda }_j^{\left(F\right)},\hfill & j=0,1,\dots ,\lceil \frac{n-1}{2}\rceil ;\hfill \\ \\ {\lambda }_{n-j}^{\left(BF\right)}=-{\lambda }_j^{\left(BF\right)},\hfill & j=1,2,\dots \lfloor \frac{n-1}{2}\rfloor .\hfill \end{array}\end{array}$$

(ii) 

Let $B\in {\mathcal{B}}_n$, $B=$ rcirc$\left(\mathbf{b}\right)$, and $F\in {\mathcal{F}}_n$, $F=$ rcirc$\left(\mathbf{f}\right)$. Then, $FB\in {\mathcal{A}}_n$, and the eigenvalues of $FB$ are expressed by

$$\begin{array}{c}\hfill \begin{array}{cc}{\lambda }_j^{\left(FB\right)}=-\mathrm{i}{\lambda }_j^{\left(B\right)}{\lambda }_j^{\left(F\right)},\hfill & j=0,1,\dots ,\lceil \frac{n-1}{2}\rceil ;\hfill \\ \\ {\lambda }_{n-j}^{\left(FB\right)}=-{\lambda }_j^{\left(FB\right)},\hfill & j=1,2,\dots \lfloor \frac{n-1}{2}\rfloor .\hfill \end{array}\end{array}$$

(iii) 

Let $A\in {\mathcal{A}}_n$, $A=$ circ$\left(\mathbf{a}\right)$ and $B\in {\mathcal{B}}_n$, $B=$ rcirc$\left(\mathbf{b}\right)$. Then, $AB\in {\mathcal{F}}_n$, and the eigenvalues of $AB$ are expressed by

$$\begin{array}{c}\hfill \begin{array}{cc}{\lambda }_j^{\left(AB\right)}=-\mathrm{i}{\lambda }_j^{\left(A\right)}{\lambda }_j^{\left(B\right)},\hfill & j=0,1,\dots ,\lceil \frac{n-1}{2}\rceil ;\hfill \\ \\ {\lambda }_{n-j}^{\left(AB\right)}=-{\lambda }_j^{\left(AB\right)},\hfill & j=1,2,\dots \lfloor \frac{n-1}{2}\rfloor .\hfill \end{array}\end{array}$$

(iv) 

Let $B\in {\mathcal{B}}_n$, $B=$ rcirc$\left(\mathbf{b}\right)$, and $A\in {\mathcal{A}}_n$, $A=$ circ$\left(\mathbf{a}\right)$. Then, $BA\in {\mathcal{F}}_n$, and the eigenvalues of $BA$ are given by

$$\begin{array}{c}\hfill \begin{array}{cc}{\lambda }_j^{\left(BA\right)}=-\mathrm{i}{\lambda }_j^{\left(B\right)}{\lambda }_j^{\left(A\right)},\hfill & j=0,1,\dots ,\lceil \frac{n-1}{2}\rceil ;\hfill \\ \\ {\lambda }_{n-j}^{\left(BA\right)}=-{\lambda }_j^{\left(BA\right)},\hfill & j=1,2,\dots \lfloor \frac{n-1}{2}\rfloor .\hfill \end{array}\end{array}$$

(v) 

Let $A\in {\mathcal{A}}_n$, $A=$ circ$\left(\mathbf{a}\right)$ and $F\in {\mathcal{F}}_n$, $F=$ rcirc$\left(\mathbf{f}\right)$. Then, $AF\in {\mathcal{B}}_n$, and the eigenvalues of $AF$ are expressed by

$$\begin{array}{c}\hfill \begin{array}{cc}{\lambda }_j^{\left(AF\right)}=-\mathrm{i}{\lambda }_j^{\left(A\right)}{\lambda }_j^{\left(F\right)},\hfill & j=0,1,\dots ,\lceil \frac{n-1}{2}\rceil ;\hfill \\ \\ {\lambda }_{n-j}^{\left(AF\right)}=-{\lambda }_j^{\left(AF\right)},\hfill & j=1,2,\dots \lfloor \frac{n-1}{2}\rfloor .\hfill \end{array}\end{array}$$

(vi) 

Let $A\in {\mathcal{A}}_n$, $A=$ circ$\left(\mathbf{a}\right)$ and $F\in {\mathcal{F}}_n$, $F=$ rcirc$\left(\mathbf{f}\right)$. Then, $FA\in {\mathcal{B}}_n$, and the eigenvalues of $FA$ are given by

$$\begin{array}{c}\hfill \begin{array}{cc}{\lambda }_j^{\left(FA\right)}=-\mathrm{i}{\lambda }_j^{\left(F\right)}{\lambda }_j^{\left(A\right)},\hfill & j=0,1,\dots ,\lceil \frac{n-1}{2}\rceil ;\hfill \\ \\ {\lambda }_{n-j}^{\left(FA\right)}=-{\lambda }_j^{\left(FA\right)},\hfill & j=1,2,\dots \lfloor \frac{n-1}{2}\rfloor .\hfill \end{array}\end{array}$$

Proof. 

See [45]. □

Note that, given $A\in {\mathcal{A}}_n$ and $B\in {\mathcal{B}}_n$, we have that ${\lambda }_j^{\left(AB\right)}=-{\lambda }_j^{\left(BA\right)}$, hence, $AB=-BA$; if $A\in {\mathcal{A}}_n$ and $F\in {\mathcal{F}}_n$, then ${\lambda }_j^{\left(AF\right)}=-{\lambda }_j^{\left(FA\right)}$, so, $AF=-FA$. Moreover, observe that, if $B_1$, $B_2\in {\mathcal{B}}_n$, $F_1$, $F_2\in {\mathcal{F}}_n$, $A_1$, $A_2\in {\mathcal{A}}_n$, then $B_1{B}_2$, $F_1{F}_2$, $A_1{A}_2\in {\mathcal{C}}_n$.

Now, we see when a $\beta$-matrix is invertible by another $\beta$-matrix.

Theorem 4.

Given $V_1\in {\mathcal{V}}_n$, $V_1=C_1+B_1+F_1+A_1$, with $C_1\in {\mathcal{C}}_n$, $B_1\in {\mathcal{B}}_n$, $F_1\in {\mathcal{F}}_n$, $A_1\in {\mathcal{A}}_n$, set ${\sigma }_j^{\left(A_1\right)}=-\mathrm{i}{\lambda }_j^{\left(A_1\right)}$, $j=0,1,\dots ,\lceil \frac{n-1}{2}\rceil$. If the matrices

$${\mathsf{\Theta }}_j=\left(\begin{array}{cccc}{\lambda }_j^{\left(C_1\right)}& {\lambda }_j^{\left(B_1\right)}& {\lambda }_j^{\left(F_1\right)}& -{\sigma }_j^{\left(A_1\right)}\\ \\ {\lambda }_j^{\left(B_1\right)}& {\lambda }_j^{\left(C_1\right)}& {\sigma }_j^{\left(A_1\right)}& -{\lambda }_j^{\left(F_1\right)}\\ \\ {\lambda }_j^{\left(F_1\right)}& -{\sigma }_j^{\left(A_1\right)}& {\lambda }_j^{\left(C_1\right)}& -{\lambda }_j^{\left(B_1\right)}\\ \\ {\sigma }_j^{\left(A_1\right)}& -{\lambda }_j^{\left(F_1\right)}& {\lambda }_j^{\left(B_1\right)}& {\lambda }_j^{\left(C_1\right)}\end{array}\right)\in {\mathbb{R}}^{4\times 4},$$

$j=0,1,\dots ,\lceil \frac{n-1}{2}\rceil$, are invertible, then there exists $V_2\in {\mathcal{V}}_n$ such that $V_1{V}_2=I_n$.

Proof. 

First of all, note that if $V_2\in {\mathcal{V}}_n$, then $V_2=C_2+B_2+F_2+A_2$, with $C_2\in {\mathcal{C}}_n$, $B_2\in {\mathcal{B}}_n$, $F_2\in {\mathcal{F}}_n$, $A_2\in {\mathcal{A}}_n$. Observe that, by Lemma 1, $V_1{V}_2=C_3+B_3+F_3+A_3$, where

$$\begin{array}{ccc}\hfill C_3& =& C_1{C}_2+B_1{B}_2+F_1{F}_2+A_1{A}_2\in {\mathcal{C}}_n,\hfill \\ \hfill B_3& =& C_1{B}_2+B_1{C}_2+F_1{A}_2+A_1{F}_2\in {\mathcal{B}}_n,\hfill \\ \hfill F_3& =& C_1{F}_2+F_1{C}_2+B_1{A}_2+A_1{B}_2\in {\mathcal{F}}_n,\hfill \\ \hfill A_3& =& C_1{A}_2+A_1{C}_2+B_1{F}_2+F_1{B}_2\in {\mathcal{A}}_n.\hfill \end{array}$$

By imposing $C_3=I_n$, we obtain

$$\begin{array}{c}\hfill {\lambda }_j^{\left(C_1\right)}{\lambda }_j^{\left(C_2\right)}+{\lambda }_j^{\left(B_1\right)}{\lambda }_j^{\left(B_2\right)}+{\lambda }_j^{\left(F_1\right)}{\lambda }_j^{\left(F_2\right)}+{\lambda }_j^{\left(A_1\right)}{\lambda }_j^{\left(A_2\right)}=1\end{array}$$

for $j=0,1,\dots ,\lceil \frac{n-1}{2}\rceil$.

Moreover, by imposing $B_3=O_n$, by virtue of Lemma 1 $\left(v\right)$ and $\left(vi\right)$, it follows that

$$\begin{array}{c}\hfill {\lambda }_j^{\left(B_1\right)}{\lambda }_j^{\left(C_2\right)}+{\lambda }_j^{\left(C_1\right)}{\lambda }_j^{\left(B_2\right)}-\mathrm{i}{\lambda }_j^{\left(A_1\right)}{\lambda }_j^{\left(F_2\right)}+\mathrm{i}{\lambda }_j^{\left(F_1\right)}{\lambda }_j^{\left(A_2\right)}=0\end{array}$$

for $j=0,1,\dots ,\lceil \frac{n-1}{2}\rceil$.

Furthermore, we impose $F_3=O_n$. Then, from Lemma 1$\left(iii\right)$ and $\left(iv\right)$, it follows that

$$\begin{array}{c}\hfill {\lambda }_j^{\left(F_1\right)}{\lambda }_j^{\left(C_2\right)}+\mathrm{i}{\lambda }_j^{\left(A_1\right)}{\lambda }_j^{\left(B_2\right)}+{\lambda }_j^{\left(C_1\right)}{\lambda }_j^{\left(F_2\right)}+\mathrm{i}{\lambda }_j^{\left(B_1\right)}{\lambda }_j^{\left(A_2\right)}=0\end{array}$$

for $j=0,1,\dots ,\lceil \frac{n-1}{2}\rceil$.

Finally, by imposing $A_3=O_n$, from Lemma 1$\left(i\right)$ and $\left(ii\right)$, we obtain

$$\begin{array}{c}\hfill {\lambda }_j^{\left(A_1\right)}{\lambda }_j^{\left(C_2\right)}-\mathrm{i}{\lambda }_j^{\left(F_1\right)}{\lambda }_j^{\left(B_2\right)}+\mathrm{i}{\lambda }_j^{\left(B_1\right)}{\lambda }_j^{\left(F_2\right)}+{\lambda }_j^{\left(C_1\right)}{\lambda }_j^{\left(A_2\right)}=0\end{array}$$

for $j=0,1,\dots ,\lceil \frac{n-1}{2}\rceil$.

Now, put ${\sigma }_j^{\left(A_2\right)}=-\mathrm{i}{\lambda }_j^{\left(A_2\right)}$, $j=0,1,\dots ,\lceil \frac{n-1}{2}\rceil$, ${{\mathbf{\vartheta }}_j}^T=\left({\lambda }_j^{\left(C_2\right)}{\lambda }_j^{\left(B_2\right)}{\lambda }_j^{\left(F_2\right)}{\sigma }_j^{\left(A_2\right)}\right)$. Since ${\mathsf{\Theta }}_j$ is invertible, then the system ${\mathsf{\Theta }}_j{\mathbf{\vartheta }}_j={\left(1000\right)}^T$ has a unique solution. □

4.4. Approximation of Symmetric Toeplitz Matrices

For each $n\in \mathbb{N}$, let us consider the following class:

$$\begin{array}{c}\hfill {\mathcal{T}}_n=\{T_n\in {\mathbb{R}}^{n\times n}:t_{k,j}=t_{|k-j|},k,j\in \{0,1,\dots ,n-1\}\}.\end{array}$$

(13)

Observe that the class defined in (13) coincides with the family of all real symmetric Toeplitz matrices.

Now, we consider the following problem: Given $T_n\in {\mathcal{T}}_n$, find

$$\begin{array}{c}\hfill V_n\left(T_n\right)=\underset{V\in {\mathcal{V}}_n}{min}{\parallel V-T_n\parallel }_F,\end{array}$$

where ${\parallel ·\parallel }_F$ denotes the Frobenius norm. It is not difficult to see that, since $T_n$ is symmetric, then we can assume that $V_n\left(T_n\right)$ is symmetric. Therefore, $V_n\left(T_n\right)=C_n\left(T_n\right)+B_n\left(T_n\right)+F_n\left(T_n\right)$, where $C_n\left(T_n\right)\in {\mathcal{C}}_n$, $B_n\left(T_n\right)\in {\mathcal{B}}_n$, and $F_n\left(T_n\right)\in {\mathcal{F}}_n$.

As regards $\gamma$-matrices, we prove the following:

Theorem 5.

Let ${\widehat{\mathcal{G}}}_n={\mathcal{S}}_n+{\mathcal{H}}_{n,1}$. Given $T_n\in {\mathcal{T}}_n$, one has

$$\begin{array}{c}\hfill G_n\left(T_n\right)=C_n\left(T_n\right)+B_n\left(T_n\right)=\underset{G\in {\widehat{\mathcal{G}}}_n}{min}\parallel G-T_n{\parallel }_F=\underset{G\in {\mathcal{G}}_n}{min}{\parallel G-T_n\parallel }_F,\end{array}$$

(14)

where $C_n\left(T_n\right)=$ circ$\left(\mathbf{c}\right)$, with

$$\begin{array}{c}\hfill c_j={\displaystyle \frac{(n-j)t_j+j{t}_{n-j}}{n}},j\in \{1,2,\dots ,n-1\};\end{array}$$

$$\begin{array}{c}\hfill c_0=t_0,\end{array}$$

and $B_n\left(T_n\right)=$ rcirc$\left(\mathbf{b}\right)$, where: for n even and $j\in \{1,2,\dots ,n-1\}\setminus \{n/2\}$,

$$\begin{array}{ccc}\hfill b_j& =& {\displaystyle \frac{1}{2n}}\left({\displaystyle \frac{4j-2n}{n}}(t_j-t_{n-j})+4\sum _{k=1}^{(j-3)/2}{\displaystyle \frac{2k+1}{n}}(t_{2k+1}-t_{n-2k-1})\right.\hfill \\ & +& \left.4\sum _{k=1}^{(n-j-3)/2}{\displaystyle \frac{2k+1}{n}}(t_{2k+1}-t_{n-2k-1})\right),\hfill \end{array}$$

j odd;

$$\begin{array}{c}\hfill b_j={\displaystyle \frac{1}{2n}}\left({\displaystyle \frac{4j-2n}{n}}(t_j-t_{n-j})+4\sum _{k=1}^{j/2-1}{\displaystyle \frac{2k}{n}}(t_{2k}-t_{n-2k})+4\sum _{k=1}^{(n-j)/2-1}{\displaystyle \frac{2k}{n}}(t_{2k}-t_{n-2k})\right),\end{array}$$

j even; for n even,

$$\begin{array}{ccc}\hfill b_0& =& {\displaystyle \frac{2}{n}}\left(\sum _{k=1}^{n/2-1}{\displaystyle \frac{2k}{n}}(t_{2k}-t_{n-2k})\right),\hfill \end{array}$$

$$\begin{array}{ccc}\hfill b_{n/2}& =& {\displaystyle \frac{4}{n}}\left(\sum _{k=1}^{n/4-1}{\displaystyle \frac{2k}{n}}(t_{2k}-t_{n-2k})\right);\hfill \end{array}$$

for n odd and $j\in \{1,2,\dots ,n-1\}$,

$$\begin{array}{ccc}\hfill b_j& =& {\displaystyle \frac{1}{2n}}\left({\displaystyle \frac{4j-2n}{n}}(t_j-t_{n-j})+4\sum _{k=0}^{(j-3)/2}{\displaystyle \frac{2k+1}{n}}(t_{2k+1}-t_{n-2k-1})\right.\hfill \\ & +& \left.4\sum _{k=1}^{(n-j)/2-1}{\displaystyle \frac{2k}{n}}(t_{2k}-t_{n-2k})\right),\hfill \end{array}$$

j odd;

$$\begin{array}{ccc}\hfill b_j& =& {\displaystyle \frac{1}{2n}}\left({\displaystyle \frac{4j-2n}{n}}(t_j-t_{n-j})+4\sum _{k=1}^{j/2-1}{\displaystyle \frac{2k}{n}}(t_{2k}-t_{n-2k})\right.\hfill \\ & +& \left.4\sum _{k=0}^{(n-j-3)/2}{\displaystyle \frac{2k+1}{n}}(t_{2k+1}-t_{n-2k-1})\right),\hfill \end{array}$$

j even; for n odd,

$$\begin{array}{ccc}\hfill b_0& =& {\displaystyle \frac{2}{n}}\left(\sum _{k=0}^{(n-3)/2}{\displaystyle \frac{2k+1}{n}}(t_{2k+1}-t_{n-2k-1})\right).\hfill \end{array}$$

Proof. 

Let us define

$$\begin{array}{c}\hfill \varphi (\mathbf{c},\mathbf{b})=\parallel T_n{-\mathrm{circ}\left(\mathbf{c}\right)-\mathrm{circ}\left(\mathbf{b}\right)\parallel }_F^2\end{array}$$

for any two symmetric vectors $\mathbf{c}$, $\mathbf{b}\in {\mathbb{R}}^n$. The proof is achieved by calculating the partial derivatives of the function $\varphi$, for details, see [45]. □

We prove an analogous result for generic $\beta$-matrices.

Theorem 6.

Given $T_n\in {\mathcal{T}}_n$, one has

$$\begin{array}{c}\hfill V_n\left(T_n\right)=C_n\left(T_n\right)+B_n\left(T_n\right)+F_n\left(T_n\right)=\underset{V\in {\mathcal{V}}_n}{min}{\parallel V-T_n\parallel }_F,\end{array}$$

(15)

where $C_n\left(T_n\right)$ and $B_n\left(T_n\right)$ are the same as those given in Theorem 5, and $F_n\left(T_n\right)=$ rcirc$\left(\mathbf{f}\right)$, where:

$$\begin{array}{c}\hfill f_j={\displaystyle \frac{t_j-t_{n-j}}{n}},j\in \{1,2,\dots ,n-1\};\end{array}$$

$$\begin{array}{c}\hfill f_0=0.\end{array}$$

Proof. 

Set

$$\begin{array}{c}\hfill \tilde{\varphi }(\mathbf{c},\mathbf{b},\mathbf{f})={\parallel T_n-\mathrm{circ}\left(\mathbf{c}\right)-\mathrm{rcirc}\left(\mathbf{b}\right)-\mathrm{rcirc}\left(\mathbf{f}\right)\parallel }_F^2\end{array}$$

for each symmetric vector $\mathbf{c}\in {\mathbb{R}}^n$, $\mathbf{b}\in {\mathbb{R}}^n$ and for every asymmetric vector $\mathbf{f}\in {\mathbb{R}}^n$. The proof is achieved by calculating the partial derivatives of the function $\varphi$, for details, see [49]. □

Note that

$$\begin{array}{c}\hfill C_n\left(T_n\right)=\underset{C\in {\mathcal{C}}_n}{min}\parallel C-T_n{\parallel }_F,H_n\left(T_n\right)=C_n\left(T_n\right)+F_n\left(T_n\right)=\underset{H\in {\mathcal{H}}_n}{min}{\parallel H-T_n\parallel }_F,\end{array}$$

(16)

where $C_n\left(T_n\right)$ is the same as the one given in Theorem 5 (see [50]), and $F_n\left(T_n\right)$ is the one given in Theorem 6 (see [19]).

Now, we show how the approximations found by $\beta$-matrices or $\gamma$-matrices allows to obtain preconditioned linear symmetric Toeplitz systems with eigenvalues clustered around 1. For every $n\in \mathbb{N}$, set

$$\begin{array}{ccc}\hfill {\widehat{\mathcal{T}}}_n& =& \{t\in {\mathcal{T}}_n:\mathrm{there}\mathrm{is}\mathrm{a}\mathrm{function}f\left(z\right)=\sum _{j=-\infty }^{+\infty }{t}_j{z}^j,\hfill \\ & & \mathrm{with}z\in \mathbb{C},\left|z\right|=1,\mathrm{and}\mathrm{such}\mathrm{that}\sum _{j=-\infty }^{+\infty }|t_j|<+\infty \}.\hfill \end{array}$$

(17)

Observe that any function f defined by a power series as in the first line of (17) is real-valued, and the set of such functions satisfying the condition ${\sum }_{j=-\infty }^{+\infty }\left|t_j\right|<+\infty$ is called Wiener class (see, e.g., [19]). Given a function f belonging to the Wiener class and a matrix $T_n\left(f\right)={\left(t_{k,j}\right)}_{k,j}\in \widehat{{\mathcal{T}}_n}$ such that $t_{k,j}=t_{|k-j|}$, $k,j\in \{0$, $1,\dots ,n-1\}$, and $f\left(z\right)={\sum }_{j=-\infty }^{+\infty }{t}_j{z}^j$, then we say that $T_n\left(f\right)$ is generated by f.

Theorem 7.

For $n\in \mathbb{N}$, given $T_n\left(f\right)\in {\widehat{\mathcal{T}}}_n$, let $C_n\left(f\right)=C_n\left(T_n\left(f\right)\right)$, $B_n\left(f\right)=B_n\left(T_n\left(f\right)\right)$, $F_n\left(f\right)=F_n\left(T_n\left(f\right)\right)$ be as in Theorems 5 and 6, and set $V_n\left(f\right)=C_n\left(f\right)+B_n\left(f\right)+F_n\left(f\right)$, and $G_n\left(f\right)=C_n\left(f\right)+B_n\left(f\right)$. Then, the following statements hold.

(i) 

For every $\epsilon >0$, there is a positive integer $n_0$, such that for each $n\ge n_0$ and for every eigenvalue ${\lambda }_j^{\left(V_n\left(f\right)\right)}$ of $V_n\left(f\right)$, it is

$$\begin{array}{c}\hfill {\lambda }_j^{\left(V_n\left(f\right)\right)}\in [f_{\min }-\epsilon ,f_{\max }+\epsilon ],j\in \{0,1,\dots ,n-1\},\end{array}$$

where $f_{\min }$ and $f_{\max }$ denote the minimum and the maximum value of f, respectively.

(ii) 

For every $\epsilon >0$, there is a positive integer $n_0$, such that for each $n\ge n_0$ and for every eigenvalue ${\lambda }_j^{\left(G_n\left(f\right)\right)}$ of $G_n\left(f\right)$, it is

$$\begin{array}{c}\hfill {\lambda }_j^{\left(G_n\left(f\right)\right)}\in [f_{\min }-\epsilon ,f_{\max }+\epsilon ],j\in \{0,1,\dots ,n-1\},\end{array}$$

where $f_{\min }$ and $f_{\max }$ denote the minimum and the maximum value of f, respectively.

(iii) 

If $V_n\left(f\right)$ is invertible, then for every $\epsilon >0$, there are k, $n_1\in \mathbb{N}$ such that for each $n\ge n_1$, the number of eigenvalues ${\lambda }_j^{\left({\left(V_n\left(f\right)\right)}^{-1}{T}_n\left(f\right)\right)}$ of $V_n^{-1}\left(f\right)T_n\left(f\right)$ such that $|{\lambda }_j^{\left({\left(V_n\left(f\right)\right)}^{-1}{T}_n\left(f\right)\right)}-1|>\epsilon$ is less than k, namely, the spectrum of ${\left(V_n\left(f\right)\right)}^{-1}{T}_n\left(f\right)$ is clustered around 1.

(iv) 

If $G_n\left(f\right)$ is invertible, then for every $\epsilon >0$ there are k, $n_1\in \mathbb{N}$ such that for each $n\ge n_1$ the number of eigenvalues ${\lambda }_j^{\left({\left(G_n\left(f\right)\right)}^{-1}{T}_n\left(f\right)\right)}$ of $G_n^{-1}\left(f\right)T_n\left(f\right)$ such that $|{\lambda }_j^{\left({\left(G_n\left(f\right)\right)}^{-1}{T}_n\left(f\right)\right)}-1|>\epsilon$ is less than k, namely, the spectrum of ${\left(G_n\left(f\right)\right)}^{-1}{T}_n\left(f\right)$ is clustered around 1.

Proof. 

For $\left(i\right)$ and $\left(iii\right)$, see [49]; for $\left(ii\right)$ and $\left(iv\right)$, see [45]. □

This result confirms how both $\beta$-matrices and $\gamma$-matrices can approximate symmetrical Toeplitz matrices well.

4.5. Choice of the Blur Matrix Approximation

In order to test the goodness of the proposed approximations, we have proceeded as follows: fixing the dimension n and the range of values which the entries of a considered Toeplitz matrices can assume, we created 10,000 different instances of Toeplitz symmetric matrices $T_n$, whose entries were randomly and uniformly chosen in the prefixed range. Then, we computed $G_n\left(T_n\right)$, $V_n\left(T_n\right)$, $C_n\left(T_n\right)$, and $H_n\left(T_n\right)$, given in (14)–(16). Then, we computed the mean of the Frobenius norm of the difference between the matrices $T_n$ and the approximating matrices. The considered range in

Table 1. Mean error obtained by the various approximations with respect to 10,000 instances of randomly generated Toeplitz matrices $T_n$ with entries in $[0,1]$.

	$\parallel {\mathit{T}}_{\mathit{n}}-{\mathit{C}}_{\mathit{n}}\left({\mathit{T}}_{\mathit{n}}\right){\parallel }_{\mathit{F}}$	$\parallel {\mathit{T}}_{\mathit{n}}-{\mathit{H}}_{\mathit{n}}\left({\mathit{T}}_{\mathit{n}}\right){\parallel }_{\mathit{F}}$	$\parallel {\mathit{T}}_{\mathit{n}}-{\mathit{G}}_{\mathit{n}}\left({\mathit{T}}_{\mathit{n}}\right){\parallel }_{\mathit{F}}$	$\parallel {\mathit{T}}_{\mathit{n}}-{\mathit{V}}_{\mathit{n}}\left({\mathit{T}}_{\mathit{n}}\right){\parallel }_{\mathit{F}}$
$n=20$	3.1389	3.1156	3.0770	3.0532
$n=25$	4.1076	4.0885	3.9591	3.9392
$n=30$	4.8062	4.7903	4.7369	4.7207
$n=35$	5.7528	5.7390	5.5989	5.5847
$n=40$	6.4536	6.4416	6.3811	6.3689
$n=45$	7.4243	7.4135	7.2649	7.2538
$n=50$	8.1211	8.1114	8.0471	8.0373
$n=100$	16.46786	16.46293	16.38939	16.38444
$n=1000$	166.48101	166.48051	166.39821	166.39771

is $[0,1]$. Note that the approximations given via $\beta$-matrices are always the best since the class ${\mathcal{V}}_n$ contains the other three classes considered. We focus on figuring out which class of matrices gives results most similar to those obtained via $\beta$-matrices. In this case, $G_n\left(T_n\right)$ is the second-best approximation in the mean.

In

Table 2. Mean error obtained by the various approximations concerning 10,000 instances of randomly generated Toeplitz matrices $T_n$ with entries in $[-1,1]$.

	$\parallel {\mathit{T}}_{\mathit{n}}-{\mathit{C}}_{\mathit{n}}\left({\mathit{T}}_{\mathit{n}}\right){\parallel }_{\mathit{F}}$	$\parallel {\mathit{T}}_{\mathit{n}}-{\mathit{H}}_{\mathit{n}}\left({\mathit{T}}_{\mathit{n}}\right){\parallel }_{\mathit{F}}$	$\parallel {\mathit{T}}_{\mathit{n}}-{\mathit{G}}_{\mathit{n}}\left({\mathit{T}}_{\mathit{n}}\right){\parallel }_{\mathit{F}}$	$\parallel {\mathit{T}}_{\mathit{n}}-{\mathit{V}}_{\mathit{n}}\left({\mathit{T}}_{\mathit{n}}\right){\parallel }_{\mathit{F}}$
$n=20$	6.2564	6.2098	6.1313	6.0838
$n=25$	8.2016	8.1633	7.8982	7.8584
$n=30$	9.6160	9.5842	9.4776	9.4453
$n=35$	11.517	11.489	11.210	11.182
$n=40$	12.915	12.891	12.771	12.747
$n=45$	14.835	14.813	14.521	14.499
$n=50$	16.292	16.272	16.141	16.121
$n=100$	32.92819	32.91833	32.76966	32.75976
$n=1000$	332.72496	332.72396	332.56154	332.56054

, the considered interval is $[-1,1]$, and the obtained results are analogous to the previous ones. In

Table 3. Mean error obtained by the various approximations concerning 10,000 instances of randomly generated Toeplitz matrices $T_n$ with decreasing entries in $[0,1]$.

	$\parallel {\mathit{T}}_{\mathit{n}}-{\mathit{C}}_{\mathit{n}}\left({\mathit{T}}_{\mathit{n}}\right){\parallel }_{\mathit{F}}$	$\parallel {\mathit{T}}_{\mathit{n}}-{\mathit{H}}_{\mathit{n}}\left({\mathit{T}}_{\mathit{n}}\right){\parallel }_{\mathit{F}}$	$\parallel {\mathit{T}}_{\mathit{n}}-{\mathit{G}}_{\mathit{n}}\left({\mathit{T}}_{\mathit{n}}\right){\parallel }_{\mathit{F}}$	$\parallel {\mathit{T}}_{\mathit{n}}-{\mathit{V}}_{\mathit{n}}\left({\mathit{T}}_{\mathit{n}}\right){\parallel }_{\mathit{F}}$
$n=20$	2.28601	2.26095	2.10745	2.08025
$n=25$	3.17788	3.15482	2.92053	2.89542
$n=30$	4.07270	4.05158	3.73644	3.71341
$n=35$	4.95798	4.93865	4.54353	4.52243
$n=40$	5.79877	5.78109	5.31037	5.29105
$n=45$	6.59117	6.57494	6.03320	6.01547
$n=50$	7.30809	7.29317	6.68763	6.67133
$n=100$	11.56697	11.55943	10.60308	10.59485
$n=1000$	13.68293	13.68225	13.43137	13.43068

, we have generated the first row of the Toeplitz symmetric matrix as follows. We set the value of the first entry equal to 1. To determine the value of the i-th entry, we multiplied the value of the $i-1$-th entry by a random constant chosen uniformly in $[0.9,1]$. Such a choice allows us to simulate better the Toeplitz matrices present in the blur operators that, in many cases, have a Gaussian shape. The behavior of the errors is similar to that of the previous cases. Moreover, it is possible to see in Table 1, Table 2 and Table 3 that, for large numbers, the $C_n\left(T_n\right)$ and $H_n\left(T_n\right)$ approximations give similar results, and that the $G_n\left(T_n\right)$ and $V_n\left(T_n\right)$ approximations are similar too.

Furthermore, as seen in

Table 4. Number of times in which the $G_n\left(T_n\right)$ approximation gives better results than the $H_n\left(T_n\right)$ approximation concerning 10,000 instances of randomly generated Toeplitz matrices $T_n$ with decreasing entries in $[0,1]$.

	$\mathit{range}=[-1,1]$	$\mathit{range}=[0,1]$
		Decreasing Case
$n=20$	8727	10,000
$n=25$	9794	10,000
$n=30$	9765	10,000
$n=35$	9973	10,000
$n=40$	9943	10,000
$n=45$	9993	10,000
$n=50$	9990	10,000
$n=100$	10,000	10,000
$n=1000$	10,000	10,000

, for large numbers, the $G_n\left(T_n\right)$ approximations are always better than the $H_n\left(T_n\right)$ approximations. Since the multiplication of $V_n\left(T_n\right)$ by a vector needs more fast discrete transforms than the multiplication of $G_n\left(T_n\right)$ by a vector, we deduce that, for n very large, $G_n\left(T_n\right)$ is the best choice considering both the quality of the approximation and the computational cost.

Thus, we define the following approximation of the energy function E in (1):

$$E^{(2+h)}\left(\mathbf{x}\right)={\parallel \mathbf{y}-\tilde{A}\mathbf{x}\parallel }^2+\sum _{c\in C}{\psi }^{\left(2\right)}(D_c\left(\mathbf{x}\right),D_{c-1}\left(\mathbf{x}\right)),$$

where $h\ge 0$ is the update step of the CATILED algorithm, and $\tilde{A}$ is the approximation of the blur matrix $\widehat{A}$, where all the symmetric Toeplitz matrices in the blocks of matrix $\widehat{A}$ are approximated using the $\gamma$-matrices given by Theorem 5. In the proposed GNC algorithm, the parameter p varies from $2+h$ to 0 with step h. We call such an algorithm E–CATILED (Extended Convex Approximation Technique for Interacting Line Elements Deblurring).

5. Experimental Results

In this section, we show, by some experimental results, how the E–CATILED algorithm achieves similar quantitative and qualitative results to CATILED, given in [5], in reduced computational time. We test the algorithms by implementing them in C language and running them in a Linux Ubuntu environment on a computer with an i5-9400F processor at 2.90 GHz. We consider both synthetic and actual data. To obtain the synthetic data, we apply to a test image a blur operator with Gaussian shape PSF (Point Spread Function) of standard deviation $\tilde{\sigma }$, and sometimes, we add an uncorrelated Gaussian noise of zero mean and variance ${\widehat{\sigma }}^2$. We use the fast transforms proposed in [38] to multiply between gamma-matrices and vectors. These transforms are explicitly designed to deal with gamma-matrices and have a small number of multiplicative operations. In the examples below, we empirically choose the involved free parameters $\widehat{\lambda }$, $\widehat{\alpha }$, and $\widehat{\epsilon }$. On the other hand, in the literature, there are available algorithms for estimating the values for the free parameters (cf. [40]).

In our first experiment, we use the ideal synthetic test image in Figure 4a. We blur this image with a Gaussian shape PSF of standard deviation $\tilde{\sigma }=1.5$, and Figure 4b presents the blurred image. Figure 4c shows the reconstruction obtained with a standard non-edge-preserving Tikhonov regularization technique (cf. [51]), where the regularization parameter $\widehat{\lambda }$ is fixed at 1. Whereas, Figure 4d shows the reconstruction obtained again by Tikhonov regularization but with $\widehat{\lambda }=0.05$. Figure 4e,f presents the results obtained with the CATILED and E–CATILED algorithms where $\widehat{\lambda }=1$, $\widehat{\alpha }=5$, and $\widehat{\epsilon }=5$. It is possible to see both quantitatively, by the MSE (Mean Squared Error) from the ideal image, and qualitatively that the images obtained with CATILED and E–CATILED are equivalent and better than those obtained with a Tikhonov regularization. In fact, by an implicit use of line elements, it is possible to obtain a more accurate reconstruction of the edges of the objects present in the ideal image. However, the computational time for determining the solution in the case of Tikhonov regularization is about one-sixth of the time of the CATILED technique. Moreover, it is possible to obtain more accurate results by minimizing the energy function in (1) via a stochastic algorithm such as simulated annealing, but with significantly longer computation times (cf. [52]).

In our next experiments, we consider a Gaussian shape PSF of standard deviation $\tilde{\sigma }=3.25$, and we apply the corresponding blurring operator to the two test images in Figure 5 to obtain the starting data. In Figure 6, we present the reconstructions obtained by CATILED and E–CATILED of the image in Figure 5a. Instead, the restorations obtained by the two algorithms of the image in Figure 5b are shown in Figure 7. In this case, we set $\widehat{\lambda }=0.05$, $\widehat{\alpha }=1$, and $\widehat{\epsilon }=1$. Figure 6d shows the reconstruction of the image in Figure 5a by a Tikhonov regularization with $\widehat{\lambda }=0.05$. Again, one can immediately see that the implicit use of line elements improves qualitatively and quantitatively the quality of the reconstructions.

In the third set of experiments, we consider a Gaussian shape PSF of standard deviation $\tilde{\sigma }=10.25$, and we add to the blurred data a Gaussian noise of variance ${\widehat{\sigma }}^2=4$. We present the reconstructions obtained by CATILED and E–CATILED for the two test images in Figure 8 and Figure 9. Here, we pose $\widehat{\lambda }=0.01$, $\widehat{\alpha }=0.1$, and $\widehat{\epsilon }=0.1$.

In

Table 5. Mean squared error of the reconstructions.

Figure	CATILED	E–CATILED
Figure 5	18.6743	18.6591
Figure 6	31.4166	31.3254
Figure 7	79.5116	79.6245
Figure 8	76.7959	76.7008
Figure 9	120.4875	120.4984

, we report the errors, in terms of MSE, of the reconstructions obtained by CATILED and E–CATILED. Thus, these experiments show that the results of the two algorithms are equivalent in both quantitative and visual terms.

Let us now consider the real data presented in Figure 10a. Such an image is an RGB color image. A color image version of CATILED is presented in [53]. However, in this case, in the blurred image, there does not appear to be any loss of saturation of the original colors. Thus, we can reconstruct each color component separately. We first assume each channel has a PFS with standard deviation $\tilde{\sigma }=5$. The reconstructions obtained by CATILED and E–CATILED are in Figure 10b,c, respectively, and the MSE between the two reconstructions is equal to 0.1320. Then, we consider a PFS with $\tilde{\sigma }=7$, and the relative results are in Figure 10d,e. Here, the MSE between the two reconstructions is 0.2615. We set for both cases $\widehat{\lambda }=0.05$, $\widehat{\alpha }=1$, and $\widehat{\epsilon }=1$. Again, the two algorithms yield qualitatively similar results.

Finally, in

Table 6. Ratios between the time costs of E–CATILED and CATILED.

Figure	Figure 4	Figure 7	Figure 6	Figure 9	Figure 8	Figure 10b,c	Figure 10d,e
Ratio	0.7356	0.8162	0.7849	0.7812	0.7598	0.7921	0.7892

, we report the ratios between the calculation times of E–CATILED and CATILED. Note here that the average computation time of the CATILED algorithm was about 96.35 min. The average computation time of the E–CATILED algorithm can be easily derived from the ratios given in Table 6. Thus, in our experimental results, using E–CATILED, we have an average computational cost gain of $22.01\%$. It is thus evident that the use of E–CATILED is more cost-effective than CATILED in terms of computational time by not affecting the quality of the reconstruction obtained.

6. Conclusions and Future Developments

In this paper, we were concerned about decreasing the computational cost of a GNC Algorithm for deblurring images when the blurring matrix is a full symmetric Toeplitz block matrix with Toeplitz blocks. We analyzed the class of $\gamma$-matrices, which are matrices for which fast transforms can perform multiplications with vectors. We showed, theoretically and experimentally, how, using $\gamma$-matrices, it is possible to obtain good approximations of symmetric Toeplitz matrices. Thus, we proposed to add a minimization of a new approximation of the energy function to the GNC technique. In that approximation, we replaced the Toeplitz matrices present in the blocks of the blur operator with $\gamma$-matrices. The experimental results show that the proposed new GNC algorithm reduces the computation time by a fifth compared with its previous version, while not changing the quality of the reconstructions. This technique could be extended in the future by considering $\gamma$-block matrices with $\gamma$-blocks and expanding the class of approximating matrices.