An Improved Convergence Theorem of the Newton-Based AOR Method for Generalized Absolute Value Equations

Abstract

For solving the large sparse generalized absolute value equations, recently a Newton-based accelerated over-relaxation (NAOR) method was investigated. In this paper, we widen the convergence regions for the parameters and establish a new convergence theorem of the NAOR method when the system matrix is an $H_{+}$-matrix. Numerical examples demonstrate that the NAOR method has a better convergence performance when the parameters are taken according to the proposed convergence theorem.

1. Introduction

Consider the generalized absolute value equation (GAVE)

$$Ax-B\left|x\right|=b,$$

(1)

where $A,B\in {\mathbb{R}}^{n\times n}$, $b\in {\mathbb{R}}^n$ and $\left|x\right|$ denotes the component-wise absolute value of the vector x. When $B=I$ with I denoting the identity matrix, the GAVE becomes the absolute value equation $Ax-\left|x\right|=b$. The GAVE was introduced by Rohn [1] and further investigated in [2,3,4]. Many problems of scientific computing and engineering applications, such as interval linear equations [5] and linear complementarity problems [6,7,8] can be equivalently transformed into the GAVE.

There exist many efficient numerical methods for the GAVE; e.g., see [9,10,11,12,13,14,15,16,17,18], and references therein. Recently, Zhou et al. proposed a Newton-based matrix splitting (NMS) method by using the matrix technique [19]. Based on the different matrix splittings, the NMS method provides a general framework of Newton-based matrix splitting methods. As a special case of the NMS method, the Newton-based accelerated over-relaxation (NAOR) method is given as follows.

Method 1 (The NAOR method [19]). Let $A=M-N$ be a splitting of the matrix A with

$$M:=\frac{1}{\alpha }(D_A-\beta L),N:=\frac{1}{\alpha }((1-\alpha )D_A+(\alpha -\beta )L+\alpha U),\alpha ,\beta \in \mathbb{R},$$

(2)

where $D_A,-L,$ and $-U$ are the diagonal, the strictly lower-triangular, and the strictly upper-triangular matrices of A, respectively. Assume that $x^{\left(0\right)}\in {\mathbb{R}}^n$ is an arbitrary initial guess. For $k=0,1,2,\dots$ until the iteration sequence ${\left\{x^{\left(k\right)}\right\}}_{k=0}^{\infty }$ is convergent, computing $x^{(k+1)}$ by

$$\begin{array}{ccc}\hfill x^{(k+1)}& =& x^{\left(k\right)}-{(M+\mathsf{\Omega })}^{-1}(A{x}^{\left(k\right)}-B|x^{\left(k\right)}|-b)\hfill \\ & =& {(M+\mathsf{\Omega })}^{-1}((N+\mathsf{\Omega })x^{\left(k\right)}+B|x^{\left(k\right)}|+b),\hfill \end{array}$$

where $M+\mathsf{\Omega }$ is invertible and $\mathsf{\Omega }$ is a given matrix.

For $\alpha =\beta \ne 0$, $\alpha =\beta =1$ and $\alpha =1,\beta =0$, the NAOR method reduces the Newton-based successive over-relaxation (NSOR) method, the Newton-based Gauss–Seidel (NGS) method and the Newton-based Jacobi (NJ) method, respectively.

From Theorem 4.5 in [19], one can observe that the associated convergence conditions have not taken into account the choice of $\mathsf{\Omega }$ and the upper bounds of $\alpha ,\beta$ are smaller than 2. However, our numerical tests reveal that better performance of the NAOR method can be obtained for $\alpha ,\beta$ larger than 2. This motivates us to look for some new convergence conditions of the NAOR method, which have the following properties:

(i)

Based on the choice of $\mathsf{\Omega }$, widen the convergence regions for the parameters $\alpha$ and $\beta$ such that their optimal values can be included inside;

(ii)

With the above new convergence conditions, the NAOR method can obtain better convergence performance.

The rest of this paper is organized as follows. In Section 2, we introduce some notations, necessary definitions, and auxiliary results. In Section 3, an improved convergence theorem for the NAOR method is proved. Two numerical experiments and some concluding remarks are given in Section 4 and Section 5, respectively.

2. Preliminaries

Some notations, definitions, and basic results are given as follows, which can be found in [20,21].

For two real $m\times n$ matrices $A=\left(a_{ij}\right)$ and $B=\left(b_{ij}\right)$, we write $A\ge B(A>B)$ if $a_{ij}\ge b_{ij}(a_{ij}>b_{ij})$ for all $1\le i\le m$ and $1\le j\le n$. A matrix $A=\left(a_{ij}\right)\in {\mathbb{R}}^{m\times n}$ is said to be nonnegative (positive) if the entries satisfy $a_{ij}\ge 0(a_{ij}>0)$ for all $1\le i\le m$ and $1\le j\le n$. Let $\left|A\right|=\left(\right|a_{ij}\left|\right)\in {\mathbb{R}}^{m\times n}$ be the absolute of the matrix A, and $A^T$ be its transpose. These notations apply to vectors in ${\mathbb{R}}^n$ as well.

Let A be a square matrix and $\rho \left(A\right)$ be its spectral radius. Its comparison matrix $⟨A⟩=\left(⟨a_{ij}⟩\right)$ is defined by $⟨a_{ij}⟩=\left|a_{ij}\right|$ if $i=j$ and $⟨a_{ij}⟩=-\left|a_{ij}\right|$ if $i\ne j$. The matrix A is called a Z-matrix if all of its off-diagonal entries are non-positive, an M-matrix if it is a Z-matrix with $A^{-1}\ge 0$, and an H-matrix if its comparison matrix $⟨A⟩$ is an M-matrix. An H-matrix with positive diagonal entries is called an $H_{+}$-matrix. If A is an M-matrix and B is a Z-matrix, then $B\ge A$ implies that B is an M-matrix.

Lemma 1

([20]). Let $A\in {\mathbb{R}}^{n\times n}$ be an H-matrix, then A is nonsingular and $|A^{-1}{|\le ⟨A⟩}^{-1}$.

Lemma 2

([22]). Let $A\in {\mathbb{R}}^{n\times n}$ be a Z-matrix. Then the following statements are equivalent:

(i)

A is an M-matrix;

(ii)

If the representation $A=M-N$ satisfies that $M^{-1}\ge 0,N\ge 0$, then $\rho \left(M^{-1}N\right)<1.$

3. Improved Convergence Theorem

In this section, we will establish a new convergence theorem of the NAOR method.

Theorem 1.

Let$A,B\in {\mathbb{R}}^{n\times n}$and A be an$H_{+}$-matrix with$A=D_A-L-U$satisfying$\rho :=\rho (D_A^{-1}\left(\right|L|+\left|U\right|+\left|B\right|)<1$, where$D_A$, $-L$and$-U$are the diagonal, the strictly lower-triangular and the strictly upper-triangular matrices of A, respectively. Assume that Ω is a positive diagonal matrix, then for any initial vector,

(i)

the NAOR method is convergent provided that α and β satisfy

$$0<\alpha \le 1,\mathrm{or}\alpha >1\mathrm{with}\mathsf{\Omega }\ge \frac{\alpha -1}{\alpha }{D}_A$$

(3)

and

$$0\le \beta \le \alpha \mathrm{with}\alpha \ne 0,\mathrm{or}\alpha <\beta <\frac{(\rho +1)\alpha }{2\rho },$$

(4)

respectively.

(ii)

The NSOR method is convergent for α satisfying (3);

(iii)

Both the NGS method and the NJ method are convergent.

Proof. 

We only need to verify the validity of $\left(i\right)$ since the others are special cases. Let

$$M_{\mathsf{\Omega }}:=⟨M+\mathsf{\Omega }⟩,N_{\mathsf{\Omega }}:=|N+\mathsf{\Omega }|+\left|B\right|\mathrm{and}{A}_{\mathsf{\Omega }}:=M_{\mathsf{\Omega }}-N_{\mathsf{\Omega }}.$$

where M and N are given as in (2). From Corollary 4.1 in [19], one only needs to prove

$$\rho \left({|(M+\mathsf{\Omega })}^{-1}|N_{\mathsf{\Omega }}\right)<1,$$

(5)

which is a sufficient convergence condition for the NAOR method.

Firstly, we will prove that the Z-matrix $A_{\mathsf{\Omega }}$ is an M-matrix. By substituting (2), the matrices $M_{\mathsf{\Omega }}$ and $N_{\mathsf{\Omega }}$ can be rewritten as

$$M_{\mathsf{\Omega }}=\mathsf{\Omega }+\frac{1}{\alpha }{D}_A-\frac{\beta }{\alpha }\left|L\right|,N_{\mathsf{\Omega }}=|\mathsf{\Omega }+\frac{1-\alpha }{\alpha }{D}_A|+\frac{|\alpha -\beta |}{\alpha }\left|L\right|+\left|U\right|+\left|B\right|.$$

(6)

A simple computation gives that $A_{\mathsf{\Omega }}=\tilde{\mathcal{M}}-\tilde{\mathcal{N}}$ with

$$\tilde{\mathcal{M}}=\mathsf{\Omega }+\frac{1}{\alpha }{D}_A-|\mathsf{\Omega }+\frac{1-\alpha }{\alpha }{D}_A|,\tilde{\mathcal{N}}=\frac{\beta +|\alpha -\beta |}{\alpha }\left|L\right|+\left|U\right|+\left|B\right|.$$

From (3), it follows that $\tilde{\mathcal{M}}=D_A$. Obviously, ${\tilde{\mathcal{M}}}^{-1}>0$ and $\tilde{\mathcal{N}}\ge 0.$ By Lemma 2, $A_{\mathsf{\Omega }}$ is an M-matrix if and only if $\rho \left({\tilde{\mathcal{M}}}^{-1}\tilde{\mathcal{N}}\right)<1$. For this, we distinguish the following two cases according to (4).

Case 1: The parameter $\beta$ satisfies that $0\le \beta \le \alpha$ with $\alpha \ne 0$.

For this case, it holds that $\tilde{\mathcal{N}}=\left|L\right|+\left|U\right|+\left|B\right|$. Combining (3) and the assumption $\rho <1$, we obtain

$$\rho \left({\tilde{\mathcal{M}}}^{-1}\tilde{\mathcal{N}}\right)=\rho \left(D_A^{-1}\tilde{\mathcal{N}}\right)=\rho <1.$$

Case 2: The parameter $\beta$ satisfies that $\alpha <\beta <\frac{(\rho +1)\alpha }{2\rho }$.

For this case, it follows that

$$\tilde{\mathcal{N}}=\frac{2\beta -\alpha }{\alpha }\left|L\right|+\left|U\right|+\left|B\right|\le \frac{2\beta -\alpha }{\alpha }\left(\right|L|+\left|U\right|+\left|B\right|).$$

Applying the monotonicity of the spectral radius of the nonnegative matrix and (3) gives

$$\rho \left({\tilde{\mathcal{M}}}^{-1}\tilde{\mathcal{N}}\right)=\rho \left(D_A^{-1}\tilde{\mathcal{N}}\right)\le \frac{2\beta -\alpha }{\alpha }\rho (D_A^{-1}\left(\right|L|+\left|U\right|+\left|B\right|\left)\right)=\frac{2\beta -\alpha }{\alpha }\rho ,$$

which, together with the assumption $\beta <\frac{\alpha (\rho +1)}{2\rho }$, implies that $\rho \left({\tilde{\mathcal{M}}}^{-1}\tilde{\mathcal{N}}\right)<1$.

From Cases 1–2, we have proved that $A_{\mathsf{\Omega }}$ is an M-matrix for $\alpha$ and $\beta$ satisfying (3) and (4). Together with the fact that $M_{\mathsf{\Omega }}\ge A_{\mathsf{\Omega }}$, it follows that $M_{\mathsf{\Omega }}$ is an M-matrix, i.e., $M+\mathsf{\Omega }$ is an $H_{+}$-matrix. This implies that

$$M_{\mathsf{\Omega }}^{-1}\ge 0\mathrm{and}\left|{(M+\mathsf{\Omega })}^{-1}\right|\le {⟨M+\mathsf{\Omega }⟩}^{-1}=M_{\mathsf{\Omega }}^{-1},$$

(7)

where the second inequality of (7) follows from Lemma 1. Note that $M_{\mathsf{\Omega }}^{-1}\ge 0$ and $N_{\mathsf{\Omega }}\ge 0$. From Lemma 2, we immediately obtain that $\rho \left(M_{\mathsf{\Omega }}^{-1}{N}_{\mathsf{\Omega }}\right)<1$. Moreover, the second inequality of (7) gives

$$\rho \left({|(M+\mathsf{\Omega })}^{-1}|N_{\mathsf{\Omega }}\right)\le \rho \left(M_{\mathsf{\Omega }}^{-1}{N}_{\mathsf{\Omega }}\right),$$

from which the convergence condition (5) holds. This proof is completed. □

Remark 1.

Comparing Theorem 1 with Theorem 4.5 in [19], we give the following remarks.

(i)

In the proofs of the above two theorems, the key is to prove that $A_{\mathsf{\Omega }}$ is an M-matrix. In Theorem 4.5 in [19], $A_{\mathsf{\Omega }}$ is defined as $A_{\mathsf{\Omega }}=M_{\mathsf{\Omega }}-{\widehat{N}}_{\mathsf{\Omega }}$, where $M_{\mathsf{\Omega }}$ is given by (6) and

$${\widehat{N}}_{\mathsf{\Omega }}=\mathsf{\Omega }+\frac{|1-\alpha |}{\alpha }{D}_A+\frac{|\alpha -\beta |}{\alpha }\left|L\right|+\left|U\right|+\left|B\right|.$$

(8)

Obviously,$A_{\mathsf{\Omega }}$is independent of Ω. From (8) and (6), it holds that ${\widehat{N}}_{\mathsf{\Omega }}\ge N_{\mathsf{\Omega }}$. Different from [19], we set $A_{\mathsf{\Omega }}=M_{\mathsf{\Omega }}-N_{\mathsf{\Omega }}$, which depends on Ω. It is easy to show that $M_{\mathsf{\Omega }}-N_{\mathsf{\Omega }}$ is an M-matrix if $M_{\mathsf{\Omega }}-{\widehat{N}}_{\mathsf{\Omega }}$ is an M-matrix, but not vice versa. This implies Theorem 1 may weaken the convergence conditions of the NAOR method.

(ii)

In Theorem 4.5 of [19], the convergence conditions on the parameters can be rewritten as

$$0<\beta \le \alpha <\frac{2}{1+\rho }\mathrm{or}\alpha <\beta <min\left\{\frac{(\rho +1)\alpha }{2\rho },\frac{2-(1-\rho )\alpha }{2\rho }\right\}.$$

(9)

Comparing (9) with (3), (4), we see easily that Theorem 1 gives a wider convergence region for α and β than Theorem 4.5 in [19]. Thus, the NAOR method may have better performance by choosing the appropriate values of α and β according to (3) and (4). This means that Theorem 1 improves Theorem 4.5 in [19].

4. Numerical Results

In this section, we use the first two examples in [19] to examine the effectiveness of Theorem 1 for the NSOR method from three aspects: the number of iteration steps (denoted by ‘IT’), the elapsed CPU time in seconds (denoted by ‘CPU’), and the norm of the relative residual vectors (denoted by ‘RES’). Here, ‘RES’ is defined as

$$\mathrm{RES}=\parallel A{x}^{\left(k\right)}-B|x^{\left(k\right)}{|-b\parallel }_2/{\parallel b\parallel }_2,$$

where $x^{\left(k\right)}$ is the k-th approximate solution to the GAVE. All numerical experiments were performed in a MATLAB environment with double machine precision. In our tests, all initial vectors were chosen as

$$x^{\left(0\right)}={(1,0,1,0,\dots ,1,0,\dots )}^T\in {\mathbb{R}}^n$$

and all iterations were terminated once $\mathrm{RES}<{10}^{-6}$.

Next, we tested two special linear complementarity problems (LCP), i.e., Examples 1 and 2 in [19], which was also given in [23,24]. As an important application of the GAVE, the LCP($q,\mathcal{M}$) is to find the vector z, such that

$$\mathcal{M}z+q\ge 0,z\ge 0\mathrm{and}{z}^T(\mathcal{M}z+q)=0,$$

with $\mathcal{M}\in {\mathbb{R}}^{n\times n},q\in {\mathbb{R}}^n$. Set

$$A=\mathcal{M}+I,B=\mathcal{M}-I\mathrm{and}x=\frac{1}{2}(\mathcal{M}-I)z+q,$$

(10)

the LCP($q,\mathcal{M}$) is equivalently transformed into the GAVE.

For simplicity, we use notations $\mathrm{Tridiag}(·)$ and $\mathrm{tridiag}(·)$ to denote the associated block-tridiagonal and tridiagonal matrices, respectively.

Example 1

(Zhou et al. [19], Example 1). Let m be a positive integer and $n=m^2$. Consider the LCP($q,\mathcal{M}$) with $\mathcal{M}=\widehat{\mathcal{M}}+\mu I$ and $q=-\mathcal{M}{z}^{*}\in {\mathbb{R}}^n$, where

$$\widehat{\mathcal{M}}=\mathrm{Tridiag}(-I,S,-I)\in {\mathbb{R}}^{n\times n},S=\mathrm{tridiag}(-1,4,-1)\in {\mathbb{R}}^{m\times m}$$

and $z^{*}={(1.2,1.2,\dots ,1.2,\dots )}^T\in {\mathbb{R}}^n$ is the unique solution of the LCP($q,\mathcal{M}$).

Example 2

(Zhou et al. [19], Example 2). Let m be a positive integer and $n=m^2$. Consider the LCP($q,\mathcal{M}$) with $\mathcal{M}=\widehat{\mathcal{M}}+\mu I$ and $q=-\mathcal{M}{z}^{*}\in {\mathbb{R}}^n$, where

$$\widehat{\mathcal{M}}=\mathrm{Tridiag}(-1.5I,S,-0.5I)\in {\mathbb{R}}^{n\times n},S=\mathrm{tridiag}(-1.5,4,-0.5)\in {\mathbb{R}}^{m\times m}$$

and $z^{*}={(1.2,1.2,\dots ,1.2,\dots )}^T\in {\mathbb{R}}^n$ is the unique solution of the LCP($q,\mathcal{M}$).

For Examples 1 and 2 with $\mu =-1$, since both the system matrices are M-matrices, the associated LCPs have the unique solutions by [7]. Moreover, their equivalent GAVEs (1) have also the unique solutions by (10).

According to Theorem 1, the NSOR method will converge to the unique solution $x^{*}$ for $\mathsf{\Omega }$ and $\alpha$ satisfying the conditions (3). In the numerical tests, we set $\mathsf{\Omega }=D_A$, which naturally satisfies (3). From this case, the NSOR method converges for $\alpha >0$.

To further obtain the suitable range of $\alpha$, the NSOR method is applied for the different scale problems with the changed $\alpha$. The test results are demonstrated in Figure 1. From Figure 1, one can obverse that the iteration steps depend on the values of $\alpha$ but are nearly independent of the sizes of the test problems. In particular, the iteration steps tend to stabilize when $\alpha >10$ for Example 1 and $\alpha >50$ for Example 2. On the other hand, we observe that the NSOR method can attain its minimum iteration steps with $\alpha$ near to 5 for Example 1 and 50 for Example 2, respectively. This means that the convergence regions for $\alpha$ by Theorem 4.5 in [19] are too small to contain its optimal values. Therefore, we will take $\alpha \in (0,5)$ for Example 1 and $(0,50)$ for Example 2 in the numerical test, respectively.

Figure 1. The iteration steps of the NSOR method with $\mathsf{\Omega }=D_A$ and the different $\alpha$: (a–c) for Example 1 and (d–f) for Example 2 with $n=m^2$.

The numerical results of the NSOR method with different sizes are given in Table 1, where ${\tilde{\alpha }}_{exp}$ and ${\alpha }_{exp}$ are obtained experimentally from the different convergence regions for $\alpha$ by minimizing the associated iteration steps. Based on the above analysis, ${\tilde{\alpha }}_{exp}$ is chosen in the interval $\alpha \in (0,5)$ for Example 1 and $\alpha \in (0,50)$ for Example 2, respectively. For comparison, we also list the results in [19], where $\mathsf{\Omega }=\widehat{\mathcal{M}}$ and ${\alpha }_{exp}$ are taken from the interval $(0,2)$.

Table 1. Numerical results of the NSOR method for Examples 1 and 2 with $\mu$ = −1.

		n	3600	4900	6400	8100	10,000
Example 1	$\mathsf{\Omega }=\widehat{M}$	${\alpha }_{exp}$	1.33	1.33	1.32	1.31	1.3
		IT	54	53	53	53	53
		CPU	0.1550	0.1654	0.2136	0.2569	0.3517
		RES	$8.60\times {10}^{-7}$	$9.65\times {10}^{-7}$	$9.27\times {10}^{-7}$	$8.92\times {10}^{-7}$	$8.57\times {10}^{-7}$
	$\mathsf{\Omega }=D_A$	${\tilde{\alpha }}_{exp}$	4.02	4.82	4.60	4.46	4.34
		IT	29	28	28	28	28
		CPU	0.0124	0.0167	0.0218	0.0284	0.0367
		RES	$9.99\times {10}^{-7}$	$9.99\times {10}^{-7}$	$9.99\times {10}^{-7}$	$9.96\times {10}^{-7}$	$9.99\times {10}^{-7}$
Example 2	$\mathsf{\Omega }=\widehat{M}$	${\alpha }_{exp}$	1.1	1.08	1.08	1.08	1.08
		IT	97	98	96	95	95
		CPU	0.2030	0.3616	0.5189	0.5863	0.7690
		RES	$9.67\times {10}^{-7}$	$9.14\times {10}^{-7}$	$9.88\times {10}^{-7}$	$8.81\times {10}^{-7}$	$9.01\times {10}^{-7}$
	$\mathsf{\Omega }=D_A$	${\tilde{\alpha }}_{exp}$	43.42	39.96	38.23	39.60	41.50
		IT	74	73	72	71	70
		CPU	0.0314	0.0445	0.0589	0.0779	0.0968
		RES	$1.00\times {10}^{-6}$	$1.00\times {10}^{-6}$	$1.00\times {10}^{-6}$	$1.00\times {10}^{-6}$	$1.00\times {10}^{-6}$

From Figure 1 and Table 1, we have the following observations and remarks:

(1)

In terms of both the CPU times and the iteration steps, the NSOR method with $\mathsf{\Omega }=D_A$ and ${\tilde{\alpha }}_{exp}$ is always superior to the one with $\mathsf{\Omega }=\widehat{M}$ and ${\alpha }_{exp}$. This means that the proposed convergence theorem improves Theorem 4.5 in [19] by taking the suitable values of $\alpha$ in a wider convergence region. In particular, the former has much less CPU time than the latter since the matrix $\mathsf{\Omega }$ is taken as $D_A$ instead of tridiagonal matrix $\widehat{M}$ in [19].

(2)

The iteration steps are nearly independent of the scale of the test problems. Hence, a strategy of choosing ${\tilde{\alpha }}_{exp}$ or ${\alpha }_{exp}$ involves testing the small-scale problems and using them to the larger-scale problems.

5. Concluding Remarks

In this paper, we investigated the new convergence conditions of the NAOR method for solving the GAVE. By considering the relationship between $\mathsf{\Omega }$ and the parameter $\alpha$, we widened the convergence regions for the two parameters. Numerical results show that the NAOR method can obtain faster convergence when the appropriate parameters are chosen according to Theorem 1 instead of Theorem 4.5 in [19].

In numerical tests, the NSOR method can attain the convergence rate nearly independent of the scale of problems. However, it is difficult to prove independence in theory. Moreover, determining optimal parameters is still a challenging problem. We will further study these interesting topics in the future.