Self-Adaptive Alternating Direction Method of Multipliers for Image Denoising

Abstract

In this study, we introduce a novel self-adaptive alternating direction method of multipliers tailored for image denoising. Our approach begins by formulating a collaborative regularization model that upholds structured sparsity within images while delving into spatial correlations among pixels. To address the challenge of penalty parameter influence on convergence speed, we innovate by proposing a self-adaptive alternating direction method of multipliers. This adaptive technique autonomously adjusts variable penalty parameters to expedite algorithm convergence, thereby markedly boosting algorithmic performance. Through a fusion of simulations and empirical analyses, our research demonstrates that this novel methodology significantly amplifies the efficacy of denoising processes.

1. Introduction

Image denoising occupies a pivotal role within the realm of digital image processing, functioning as a foundational task crucial for refining and accentuating the intrinsic qualities inherent in an image [1,2,3,4]. Noise, stemming from a multitude of sources, such as image acquisition devices, transmission channels, or storage modalities, presents a substantial impediment to maintaining image fidelity. The presence of noise can obfuscate intricate details, introduce artifacts, and degrade the overall visual fidelity of an image. Through the efficacious elimination of noise via denoising methodologies, the quality, integrity, and interpretability of images can be significantly augmented [5,6,7,8]. In domains like medical imaging, surveillance, remote sensing, and scientific inquiry, where accurate and lucid visual information holds paramount importance, the indispensability of image denoising cannot be overstated. It not only enhances the aesthetic allure of images but also safeguards the preservation of critical details, fostering more dependable analysis, interpretation, and decision-making grounded in visual data [9,10,11].

Common image denoising techniques include statistical modeling, transformation methods, and deep learning strategies [12,13,14]. Compared to other methods, transformation methods have efficient algorithm implementations, capable of processing large-scale data in a short time. They also exhibit good adaptability to various signal types, making them suitable for a variety of image processing tasks. Low-rank matrix recovery techniques model low-rank matrices within the image domain to restore the original low-rank structure from corrupted image data [15,16,17,18]. This approach leverages the latent low-rank properties within images, representing the image as a sum of a low-rank matrix capturing structural information and a sparse matrix representing noise and fine details. Through low-rank matrix recovery techniques, noise in images can be removed while preserving essential structural features, resulting in clearer and more accurate image restoration [19].

Low-rank matrix recovery techniques include singular value soft thresholding, low-rank nuclear norm regularization, alternating direction method of multipliers, and low-rank representation methods [20,21,22]. These methods operate by manipulating singular values, introducing penalties based on low-rank nuclear norms, employing alternating optimization schemes, or utilizing low-rank representations to restore the low-rank structure of images while eliminating noise. In recent years, several works have garnered significant attention. For example, in [23], Liu et al. utilized sparse signal representation methods and designed overcomplete dictionaries to explore the advantages and limitations of sparse signal representation algorithms in the processing of ultrasonic non-destructive evaluation signals. In [24], Hu et al. reconstructed images from their under-sampled Fourier coefficients using infimal convolution regularizations, and an advanced Generalized Structured Low-Rank Algorithm is introduced. Due to its applicability to various constraints and regularization terms, the ADMM has garnered more attention among these low-rank matrix recovery algorithms.

The traditional ADMM methodology employs the minimization of the (1,1)-norm of a matrix to enforce sparsity and control the count of non-zero elements, thereby approximating the optimization of the matrix’s 0-norm to a certain degree. However, neighboring pixels in images typically exhibit correlations. By enforcing sparse constraints on individual pixels, there is a risk of losing model intricacies and structural nuances. Through the utilization of collaborative sparse regression, a shared sparse representation can be identified, preserving structured sparsity within the image and enabling a deeper analysis of spatial interrelations among pixels [25]. To mitigate the impact of penalty parameters on convergence rates, an adaptive approach based on a balance principle for selecting appropriate penalty parameters is introduced. This methodology, proven effective in projection techniques and Uzawa block relaxation algorithms [26,27], is leveraged in this study to propose a self-adaptive ADMM strategy. This strategy dynamically selects variable penalty parameters to expedite algorithm convergence, markedly enhancing algorithmic efficacy.

This paper aims to enhance denoising effectiveness by introducing a collaborative regularization model and proposing a self-adaptive alternating direction method of multipliers to achieve faster algorithm convergence. The key contributions are as follows:

• Formulate a collaborative regularization model that maintains structured sparsity within images and explores spatial correlations among pixels.
• Propose a self-adaptive alternating direction method of multipliers to achieve faster algorithm convergence.

The remaining part of this paper is organized as follows. In Section 2, the related works are discussed. In Section 3, low-rank matrix recovery and the alternating direction method of multipliers are introduced. In Section 4, the proposed collaborative regularization model and improved algorithm are proposed. In Section 5, two examples are provided. Finally, some concluding remarks are given in Section 6.

2. Related Work

In recent years, image denoising has become a highly regarded application area in the field of artificial intelligence. In image processing, the Low-Rank Matrix Recovery (LRMR) technique has been successfully applied to remove noise from images, enhancing image quality and clarity [28,29]. LRMR technology extends the concept of sparse representation of sample vectors to the low-rank scenario of matrices, becoming another important method for image denoising, following the path of compressive sensing (CS). In image denoising, LRMR decomposes the image data matrix into the sum of a low-rank matrix and a sparse noise matrix, then restores the low-rank matrix by solving a nuclear norm optimization problem, effectively eliminating noise from the image. Currently, LRMR mainly includes several common models such as Robust Principal Component Analysis (RPCA), Matrix Completion (MC), and Low-Rank Representation (LRR), all of which demonstrate good image denoising effects in various scenarios [30,31].

RPCA is a technique used to handle outlier values and noise in data, aiming to decompose the original data matrix into low-rank and sparse components. The core idea of RPCA assumes that the data matrix consists of a low-rank structure and a sparse noise component, recovering the original data by minimizing the difference between these two parts, thereby achieving denoising and outlier detection. RPCA exhibits strong robustness to outlier values and noise in data, effectively dealing with interfering factors, enhancing data accuracy and stability. It finds wide applications in fields such as image processing, video analysis, signal processing, and more. For example, in [32], Chen et al. proposed a robust PCA method that relies on a nonconvex low-rank approximation and total variational regularization (TV) to address the image denoising problem, aiming to enhance the denoising performance. In [33], Wu et al., proposed a workflow to reduce seismic traffic noise by utilizing the $l_p$-norm RPCA method. Although these algorithms perform well under specific common scenarios, in order to achieve real-time processing, it is necessary to accelerate and optimize the low-rank matrix recovery algorithm. This optimization involves reducing algorithm complexity, minimizing computational burden, and this is also one of the main insights of our research.

3. Preliminaries and Problem Statement

3.1. Low-Rank Matrix Recovery

The application of low-rank matrix recovery in image denoising is a significant and effective technique within the field of image processing. In the field of image processing, images are often regarded as matrices with low-rank properties due to the inherent correlations and structural information among pixels. The objective of image denoising is to eliminate noise from various sources such as sensor noise, compression artifacts, or communication channel interference to enhance image quality and clarity.

For a given matrix $\mathbf{D}$ containing noise, the matrix $\mathbf{D}$ can be decomposed into the sum of two matrices, i.e., $\mathbf{D}=\mathbf{Z}+\mathbf{X}$, where the matrices $\mathbf{Z}$ and $\mathbf{X}$ are unknown, but $\mathbf{Z}$ is low-rank. When the elements of the matrix $\mathbf{X}$ obey an independent identically distributed Gaussian distribution, the classical PCA can be used to obtain the optimal matrix $\mathbf{Z}$, i.e., to solve the following optimization problem.

$$\begin{array}{c}\hfill \underset{\mathbf{Z},\mathbf{X}}{min}{\left|\right|\mathbf{Z}\left|\right|}_F,\mathrm{s}.\mathrm{t}.\mathrm{rank}\left(\mathbf{Z}\right)\le r,\mathbf{D}=\mathbf{Z}+\mathbf{X}\end{array}$$

(1)

When $\mathbf{X}$ is a sparse large noise matrix, PCA is no longer applicable. In this case, the recovery of the low-rank matrix $\mathbf{Z}$ is a two-objective optimization problem

$$\begin{array}{c}\hfill \underset{\mathbf{Z},\mathbf{X}}{min}{(\mathrm{rank}\left(\mathbf{Z}\right),|\left|\mathbf{X}\right||}_0),\mathrm{s}.\mathrm{t}.\mathbf{D}=\mathbf{Z}+\mathbf{X}\end{array}$$

(2)

The bi-objective optimization problem (2) is converted into a single-objective optimization problem by introducing a trade-off factor $\lambda (0<\lambda <1)$ as follows:

$$\begin{array}{c}\hfill \underset{\mathbf{Z},\mathbf{X}}{min}\mathrm{rank}\left(\mathbf{Z}\right)+{\lambda \left|\right|\mathbf{X}\left|\right|}_0,\mathrm{s}.\mathrm{t}.\mathbf{D}=\mathbf{Z}+\mathbf{X}\end{array}$$

(3)

Since the objective function of Equation (3) is non-convex, it can be approximated by the following equation:

$$\begin{array}{c}\hfill \underset{\mathbf{Z},\mathbf{X}}{min}{\left|\right|\mathbf{Z}\left|\right|}_{*}+{\lambda \left|\right|\mathbf{X}\left|\right|}_{1,1},\mathrm{s}.\mathrm{t}.\mathbf{D}=\mathbf{Z}+\mathbf{X}\end{array}$$

(4)

Remark 1.

The (1,1)-norm of a matrix is defined as the maximum absolute sum of each column. In a sense, this maximum value can be seen as an upper bound on the number of non-zero elements in the matrix. Therefore, minimizing the (1,1)-norm of a matrix also serves as a constraint on the number of non-zero elements, thus providing an approximation to optimizing the 0-norm of a matrix.

Remark 2.

The problem of solving Equation (4) is known as Robust Principal Component Analysis (RPCA), which provides an efficient way to solve the low-rank matrix recovery problem and can improve robustness to outliers in the data.

Remark 3.

In reference [15], it is pointed out that as long as the singular values distribution of the low-rank matrix $\mathbf{Z}$ is reasonable and the non-zero elements of the sparse matrix are uniformly distributed, the convex optimization problem (4) can recover the original low-rank matrix from any unknown error with a probability close to 1.

3.2. Traditional Alternating Direction Method of Multipliers

The Alternating Direction Method of Multipliers (ADMM) is an iterative algorithm used for solving convex optimization problems. ADMM decomposes the original problem into easier-to-handle subproblems and iteratively solves these subproblems alternately, combined with updating the multipliers to progressively approach the optimal solution of the original problem. This method excels in dealing with problems that have structural constraints and finds wide applications in distributed optimization and machine learning fields. Introducing the augmented Lagrangian function for problem (4):

$$\begin{array}{cc}\hfill L(\mathbf{Z},\mathbf{X},\mathbf{Y},\mathbf{u})& ={\left|\right|\mathbf{Z}\left|\right|}_{*}+{\lambda \left|\right|\mathbf{X}\left|\right|}_{1,1}+<\mathbf{Y},\mathbf{D}-\mathbf{X}-\mathbf{Z}>\\ & +{\displaystyle \frac{\gamma }{2}}{\left|\right|\mathbf{D}-\mathbf{Z}-\mathbf{X}\left|\right|}_F^2,\end{array}$$

(5)

where $\mathbf{Y}$ is the Lagrange multiplier and $\gamma >0$ is the given penalty parameter. Using the Exact Augmented Lagrangian Method (EALM), alternatively iterate matrices $\mathbf{Z}$ and $\mathbf{X}$,

$$\begin{array}{c}\hfill {\mathbf{Z}}_{k+1}^{j+1}=\arg \underset{\mathbf{Z}}{min}L({\mathbf{Z},\mathbf{X}}_{k+1}^j,{\mathbf{Y}}_k,u_k),\end{array}$$

(6)

$$\begin{array}{c}\hfill {\mathbf{X}}_{k+1}^{j+1}=\arg \underset{\mathbf{X}}{min}L({\mathbf{Z}}_{k+1}^{j+1},\mathbf{X},{\mathbf{Y}}_k,u_k).\end{array}$$

(7)

If ${\mathbf{Z}}_{k+1}^{*}$ and ${\mathbf{X}}_{k+1}^{*}$ represent the precise values of ${\mathbf{Z}}_{k+1}^{j+1}$ and ${\mathbf{X}}_{k+1}^{j+1}$, the formula for updating matrix $\mathbf{Y}$ is as follows:

$$\begin{array}{c}\hfill {\mathbf{Y}}_{k+1}={\mathbf{Y}}_k+u_k(\mathbf{D}-{\mathbf{Z}}_{k+1}^{*}-{\mathbf{X}}_{k+1}^{*}),\end{array}$$

(8)

where the updating method for parameter $u_k$ can be found in reference [15].

4. Proposed Model and Improved Algorithm

This section primarily outlines the refinement of our collaborative regularization model [25]. In enhancing the convergence speed while solving this model, we propose our self-adaptive alternating direction method by deriving adaptive step sizes.

4.1. A Collaborative Regularization Model

In image denoising, noise can be seen as high-frequency variations that disrupt the smoothness and continuity of the image. By representing an image as a low-rank matrix (which captures the underlying structure and patterns in the image), the high-frequency noise components can often be separated from the low-rank structure during the denoising process. Images often exhibit high levels of redundancy, meaning that the information content of an image can be effectively represented with fewer parameters than the pixel count. Low-rank matrix recovery methods leverage this redundancy by approximating the image data with a low-rank matrix, which helps in denoising by emphasizing the common structure shared among pixels.

In Equation (4), constraining the noise matrix X with the (1,1) norm can induce sparsity in the matrix, causing some pixel values to tend towards zero, thereby suppressing noise in the image. As images typically exhibit edge sparsity, the (1,1) norm of the matrix can to some extent help preserve the edge information in the image, thereby maintaining the image’s structure.

Adjacent pixels in an image commonly demonstrate similarity. To address this correlation, collaborative sparse regression can be employed to derive a shared sparse representation, thus preserving the structured sparsity within the image. By delving deeper into the spatial relationships among pixels, the objective is to induce sparsity in the interactions between neighboring pixels, thereby enhancing the preservation of the image’s structural information. Accordiong to [25], replace the (1,1) norm in Equation (4) with the (2,1) norm,

$$\begin{array}{c}\hfill \underset{\mathbf{Z},\mathbf{X}}{min}{\left|\right|\mathbf{Z}\left|\right|}_{*}+{\lambda \left|\right|\mathbf{X}\left|\right|}_{2,1},\mathrm{s}.\mathrm{t}.\mathbf{D}=\mathbf{Z}+\mathbf{X}\end{array}$$

(9)

Similarly, to solve Equation (9) using the ADMM method, the first step is to construct the Lagrangian function:

$$\begin{array}{c}\hfill L(\mathbf{Z},\mathbf{X},\mathbf{Y},u)={\left|\right|\mathbf{Z}\left|\right|}_{*}+{\lambda \left|\right|\mathbf{X}\left|\right|}_{2,1}+<\mathbf{Y},\mathbf{D}-\mathbf{X}-\mathbf{Z}>\\ \hfill +{\displaystyle \frac{\gamma }{2}}{\left|\right|\mathbf{D}-\mathbf{Z}-\mathbf{X}\left|\right|}_F^2\end{array}$$

(10)

If $\mathbf{X}={\mathbf{X}}_{k+1}^j$,

$$\begin{array}{ccc}\hfill {\mathbf{Z}}_{k+1}^{j+1}& =& \arg \underset{\mathbf{Z}}{min}L({\mathbf{Z},\mathbf{X}}_{k+1}^j,{\mathbf{Y}}_k,u_k)\hfill \\ & =& \arg \underset{\mathbf{Z}}{min}{\left|\right|\mathbf{Z}\left|\right|}_{*}+{\displaystyle \frac{u_k}{2}}\left|\right|{\mathbf{Z}-(\mathbf{D}-\mathbf{X}}_{k+1}^j+{\displaystyle \frac{{\mathbf{Y}}_k}{u_k}}{\left)\right||}_F^2\hfill \\ & =& \mathbf{U}{S}_{{\displaystyle \frac{1}{u_k}}}({\mathbf{D}-\mathbf{X}}_{k+1}^j+{\displaystyle \frac{{\mathbf{Y}}_k}{u_k}})\mathbf{V}\hfill \end{array}$$

(11)

where U$\mathsf{\Sigma }$V represents the singular value decomposition of a matrix, $S_{{\displaystyle \frac{1}{u_k}}}\left(A\right)$ represents a matrix where its (i, j) element is $max\left(\right|Aij|-ϵ,0)$ × $\mathrm{sgn}\left(Aij\right)$, where the parameter $ϵ>0$. Subsequently, update matrix $\mathbf{X}$ based on the obtained ${\mathbf{Z}}_{k+1}^{j+1}$:

$$\begin{array}{ccc}\hfill {\mathbf{X}}_{k+1}^{j+1}& =& \arg \underset{\mathbf{X}}{min}L({\mathbf{Z}}_{k+1}^{j+1},{\mathbf{X},\mathbf{Y}}_k,u_k)\hfill \\ & =& \arg \underset{\mathbf{X}}{min}{\left|\right|\mathbf{X}\left|\right|}_{1,1}+{\displaystyle \frac{u_k}{2}}\left|\right|\mathbf{X}-({\mathbf{D}-\mathbf{Z}}_{k+1}^{j+1}+{\displaystyle \frac{{\mathbf{Y}}_k}{u_k}}){\left|\right|}_F^2\hfill \end{array}$$

(12)

Remark 4.

In collaborative regularization models, the $L_1$ norm is typically used in conjunction with the $L_2$ norm, allowing the model to explore spatial correlations among pixels in the image. By modeling relationships between different pixels, the model can better capture the spatial structure and texture information in the image, thereby enhancing the accuracy and effectiveness of image processing. The collaborative regularization (2,1 norm) model combines the advantages of the $L_1$ and $L_2$ norms, maintaining the structural sparsity of the image while delving into the spatial correlations among pixels.

Equation (12) can be solved using the following lemma.

Lemma 1.

Let $\mathbf{A}=[a_1,a_2,\dots ,a_i,\dots ]$ be a given matrix and ${\left|\right|\left|\right|}_F$ be the Frobenius norm. If the optimal solution of $\underset{\mathbf{R}}{min}{\lambda \left|\right|\mathbf{R}\left|\right|}_{2,1}+{\displaystyle \frac{1}{2}}{\left|\right|\mathbf{R}-\mathbf{A}\left|\right|}_F^2$ is ${\mathbf{R}}^{*}$, then the $i_{th}$ column of ${\mathbf{R}}^{*}$ is

$$\begin{array}{c}\hfill {\mathbf{R}}^{*}(:,i)=\left\{\begin{array}{cc}{\displaystyle \frac{\left|\right|a_i\left|\right|-\lambda }{\left|\right|a_i\left|\right|}}{a}_i,\hfill & \mathrm{if}\lambda <\left|\right|a_i\left|\right|\hfill \\ 0,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.\end{array}$$

After obtaining ${\mathbf{X}}_{k+1}^{j+1}$ using Lemma 1, update $\mathbf{Y}$ as follows:

$$\begin{array}{c}\hfill {\mathbf{Y}}_{k+1}={\mathbf{Y}}_k+u_k({\mathbf{D}-\mathbf{Z}}_{k+1}^{*}-{\mathbf{X}}_{k+1}^{*}).\end{array}$$

(13)

4.2. Self-Adaptive Alternating Direction Method of Multipliers

The ADMM converges for all positive penalty parameters, usually using fixed parameters. However, the convergence speed of this method is highly dependent on the penalty parameters, making it challenging to choose suitable ones for specific problems. To address the issue of selecting appropriate penalty parameters, a refined version of the alternating direction method of multipliers is proposed. This approach leverages an adaptive rule and approximates the optimal penalty parameters based on iterative results rather than relying on fixed parameters.

Lemma 2.

The sequence ${\mathbf{Z}}_k,{\mathbf{X}}_k,{\mathbf{Y}}_k$ obtained from the ADMM satisfies

$$\begin{array}{cc}\hfill {\gamma }^2\left|\right|{\mathbf{X}}_{k+1}-{\mathbf{X}}_k{\left|\right|}_F^2+\left|\right|{\mathbf{Y}}_{k+1}-{\mathbf{Y}}_k{\left|\right|}_F^2& \le \left({\gamma }^2\right||{\mathbf{X}}_k-{\mathbf{X}}^{*}|{|}_F^2+\left|\right|{\mathbf{Y}}_k-{\mathbf{Y}}^{*}\left|{|}_F^2\right)\\ & -\left({\gamma }^2\right||{\mathbf{X}}_{k+1}-{\mathbf{X}}^{*}|{|}_F^2+\left|\right|{\mathbf{Y}}_{k+1}-{\mathbf{Y}}^{*}\left|{|}_F^2\right)\end{array}$$

In Lemma 2, replacing $\gamma$ with ${\gamma }_k$, the sequence ${\mathbf{Z}}_k,{\mathbf{X}}_k,{\mathbf{Y}}_k$ generated by ADMM satisfies

$$\begin{array}{cc}\hfill \left({\gamma }_k^2\right||{\mathbf{X}}_k-{\mathbf{X}}^{*}|{|}_F^2+\left|\right|{\mathbf{Y}}_k-Y^{*}\left|{|}_F^2\right)& -\left({\gamma }_k^2\right||{\mathbf{X}}_{k+1}-{\mathbf{X}}^{*}|{|}_F^2+\left|\right|{\mathbf{Y}}_{k+1}-{\mathbf{Y}}^{*}\left|{|}_F^2\right)\\ & \ge {\gamma }_k^2\left|\right|{\mathbf{X}}_{k+1}-{\mathbf{X}}_k{\left|\right|}_F^2+\left|\right|{\mathbf{Y}}_{k+1}-{\mathbf{Y}}_k{\left|\right|}_F^2.\end{array}$$

(14)

As a result of ${\gamma }_k^2\left|\right|{\mathbf{X}}_{k+1}-{\mathbf{X}}_k{\left|\right|}_F^2+\left|\right|{\mathbf{Y}}_{k+1}-{\mathbf{Y}}_k{\left|\right|}_F^2\ge 0$, hence

$$\begin{array}{c}\hfill ({\gamma }_k^2\left|\right|{\mathbf{X}}_k-{\mathbf{X}}^{*}{\left|\right|}_F^2+\left|\right|{\mathbf{Y}}_k-{\mathbf{Y}}^{*}{\left|\right|}_F^2)\ge ({\gamma }_k^2\left|\right|{\mathbf{X}}_{k+1}-{\mathbf{X}}^{*}{\left|\right|}_F^2+\left|\right|{\mathbf{Y}}_{k+1}-{\mathbf{Y}}^{*}{\left|\right|}_F^2).\end{array}$$

(15)

The sequence $\left({\gamma }_k^2\right||{\mathbf{X}}_k-{\mathbf{X}}^{*}|{|}_F^2+\left|\right|{\mathbf{Y}}_k-{\mathbf{Y}}^{*}\left|{|}_F^2\right)$ is evidently monotonically decreasing and bounded. In order to expedite the convergence rate, it is imperative to satisfy the following expression:

$$\begin{array}{c}\hfill {\gamma }_k^2\left|\right|{\mathbf{X}}_k-{\mathbf{X}}^{*}{\left|\right|}_F\approx \left|\right|{\mathbf{Y}}_k-{\mathbf{Y}}^{*}{\left|\right|}_F.\end{array}$$

(16)

Substitute ${\mathbf{Y}}_{k+1}$ for ${\mathbf{Y}}^{*}$ and ${\mathbf{X}}_{k+1}$ for ${\mathbf{X}}^{*}$, respectively:

$$\begin{array}{c}\hfill {\gamma }_k^2\left|\right|{\mathbf{X}}_k-{\mathbf{X}}_{k+1}{\left|\right|}_F\approx \left|\right|{\mathbf{Y}}_k-{\mathbf{Y}}_{k+1}{\left|\right|}_F.\end{array}$$

(17)

Thus, the fundamental approach to selecting the penalty parameter ${\gamma }_k$ can be derived. For a given positive constant $\rho$, if

$$\begin{array}{c}\hfill {\gamma }_k^2\left|\right|{\mathbf{X}}_k-{\mathbf{X}}_{k+1}{\left|\right|}_F>(1+\rho )\left|\right|{\mathbf{Y}}_k-{\mathbf{Y}}_{k+1}{\left|\right|}_F.\end{array}$$

(18)

then reduce ${\gamma }_k$ in the next iteration. If

$$\begin{array}{c}\hfill {\gamma }_k^2\left|\right|{\mathbf{X}}_k-{\mathbf{X}}_{k+1}{\left|\right|}_F<{\displaystyle \frac{1}{(1+\rho )}}\left|\right|{\mathbf{Y}}_k-{\mathbf{Y}}_{k+1}{\left|\right|}_F.\end{array}$$

(19)

then increase ${\gamma }_k$ in the next iteration. In conclusion, the precise guideline for selecting the penalty parameter ${\gamma }_k$ is outlined as follows:

$$\begin{array}{c}\hfill {\gamma }_{k+1}=\left\{\begin{array}{cc}(1+m_k){\gamma }_k\hfill & \mathrm{if}{\displaystyle \frac{\left|\right|{\mathbf{Y}}_k-{\mathbf{Y}}_{k+1}{\left|\right|}_F}{{\gamma }_k\left|\right|{\mathbf{X}}_k-{\mathbf{X}}_{k+1}{\left|\right|}_F}}>1+\rho \hfill \\ {\displaystyle \frac{1}{(1+m_k)}}{\gamma }_k\hfill & \mathrm{if}{\displaystyle \frac{\left|\right|{\mathbf{Y}}_k-{\mathbf{Y}}_{k+1}{\left|\right|}_F}{{\gamma }_k\left|\right|{\mathbf{X}}_k-{\mathbf{X}}_{k+1}{\left|\right|}_F}}<1+\rho \hfill \\ {\gamma }_k\hfill & \mathrm{if}{\displaystyle \frac{\left|\right|{\mathbf{Y}}_k-{\mathbf{Y}}_{k+1}{\left|\right|}_F}{{\gamma }_k\left|\right|{\mathbf{X}}_k-{\mathbf{X}}_{k+1}{\left|\right|}_F}}=1+\rho \hfill \end{array}\right.\end{array}$$

(20)

where $m_k\ge 0,{\sum }_{k=0}^{\infty }{m}_k<\infty$. $m_k$ is calculated as follows:

$$\begin{array}{c}\hfill m_k=\left\{\begin{array}{cc}\rho ,\hfill & \mathrm{if}{b}_k<b_{k+1},b_{k+1}\le b_{max}\hfill \\ {\displaystyle \frac{1}{{(b_{k+1}-b_{max})}^2}}{\gamma }_k,\hfill & \mathrm{if}{b}_k<b_{k+1},b_{k+1}\ge b_{max}\hfill \\ 0.\hfill & \mathrm{otherwise}\hfill \end{array}\right.\end{array}$$

(21)

where $b_0=0$,

$$\begin{array}{c}\hfill b_{k+1}=\left\{\begin{array}{cc}{b}_k,\hfill & \mathrm{if}{\displaystyle \frac{1}{1+\rho }}\le {\displaystyle \frac{\left|\right|{\mathbf{Y}}_k-{\mathbf{Y}}_{k+1}{\left|\right|}_F}{{\gamma }^2\left|\right|{\mathbf{X}}_k-{\mathbf{X}}_{k+1}{\left|\right|}_F}}\le 1+\rho \hfill \\ b_k+1.\hfill & \mathrm{otherwise}\hfill \end{array}\right.\end{array}$$

(22)

The steps of the S-ADMM algorithm are listed as follows.

• Initialize $\mathbf{X},\mathbf{Y},\gamma >0,{\gamma }_k=\gamma >0,k=0.$
• Calculate the value of ${\mathbf{Z}}_{k+1}$ such that it satisfies the following condition for any $\mathbf{Z}$:

  $$\begin{array}{c}\hfill {\mathbf{Z}}_{k+1}=\arg \underset{\mathbf{Z}}{min}L(\mathbf{Z},{\mathbf{X}}_{k+1},{\mathbf{Y}}_k,u_k)\end{array}$$
• Calculate the value of ${\mathbf{X}}_{k+1}$ such that it satisfies the following condition for any $\mathbf{X}$:

  $$\begin{array}{c}\hfill {\mathbf{X}}_{k+1}=\arg \underset{\mathbf{X}}{min}L({\mathbf{Z}}_{k+1},\mathbf{X},{\mathbf{Y}}_k,u_k)\end{array}$$
• Update Lagrange multiplier $\mathbf{Y}$ as follows:

  $$\begin{array}{c}\hfill {\mathbf{Y}}_{k+1}={\mathbf{Y}}_k+{\gamma }_k(\mathbf{D}-{\mathbf{Z}}_{k+1}^{*}-{\mathbf{X}}_{k+1}^{*}).\end{array}$$
• Update penalty parameter as follows:

  $$\begin{array}{c}\hfill {\gamma }_{k+1}=\left\{\begin{array}{cc}(1+m_k){\gamma }_k\hfill & \mathrm{if}{\displaystyle \frac{\left|\right|{\mathbf{Y}}_k-{\mathbf{Y}}_{k+1}{\left|\right|}_F}{{\gamma }_k\left|\right|{\mathbf{X}}_k-{\mathbf{X}}_{k+1}{\left|\right|}_F}}>1+\rho \hfill \\ {\displaystyle \frac{1}{(1+m_k)}}{\gamma }_k\hfill & \mathrm{if}{\displaystyle \frac{\left|\right|{\mathbf{Y}}_k-{\mathbf{Y}}_{k+1}{\left|\right|}_F}{{\gamma }_k\left|\right|{\mathbf{X}}_k-{\mathbf{X}}_{k+1}{\left|\right|}_F}}<1+\rho \hfill \\ {\gamma }_k\hfill & \mathrm{if}{\displaystyle \frac{\left|\right|{\mathbf{Y}}_k-{\mathbf{Y}}_{k+1}{\left|\right|}_F}{{\gamma }_k\left|\right|{\mathbf{X}}_k-{\mathbf{X}}_{k+1}{\left|\right|}_F}}=1+\rho \hfill \end{array}\right.\end{array}$$
• For a given error limit $ϵ>0$, if $\left|\right|{\mathbf{Z}}_k-{\mathbf{Z}}_{k+1}{\left|\right|}_F^2+\left|\right|{\mathbf{X}}_k-{\mathbf{X}}_{k+1}{\left|\right|}_F^2<ϵ$ is satisfied, the iteration stops, yielding the numerical solution $({\mathbf{Z}}_{k+1},{\mathbf{X}}_{k+1})$; otherwise, set $k=k+1$ and return to step 2.

5. Experiment Results

5.1. Experiment Results with Synthetic Data

In this section, through numerical simulations, we compare the effectiveness of the traditional ADMM algorithm and our improved adaptive ADMM algorithm. All the experiments are performed on the same workbench. Grayscale images are conventionally characterized by a two-dimensional matrix representation, wherein the value of each element denotes the respective pixel’s grayscale intensity. We need to generate a low-rank matrix with rank r:

$${\mathbf{Q}}_0=\mathbf{S}{\mathbf{W}}^{\mathrm{T}},$$

(23)

where $\mathbf{S}\in {\mathbf{R}}^{m\times r}$ and $\mathbf{W}\in {\mathbf{R}}^{n\times r}$ are random matrices whose elements obey a Gaussian distribution, and the rank of the matrix ${\mathbf{Q}}_0$ is r. Let the rank parameter $r_p={\displaystyle \frac{r}{m}}$. ${\mathbf{Q}}_0$ can be considered as the genuine noise-free image. Next we generate a sparse noise matrix where 20% of the data points are noise points. Hence, the image after contamination can be denoted by $\mathbf{D}={\mathbf{X}}_0+{\mathbf{Z}}_0$, where $[{\mathbf{X}}_0,{\mathbf{Z}}_0]$ denotes the true solution matrix.

We use our improved S-ADMM algorithm to compare it with the traditional ADMM algorithm. Figure 1 shows the error comparison between the S-ADMM algorithm and the ADMM algorithm at different numbers of iterations. As can be seen from the figure, the error of the S-ADMM algorithm is always smaller than that of the ADMM algorithm as the number of iterations increases.

Table 1. Errors under different numbers of iterations.

Iteration	5	10	25	100
ADMM	0.1962	0.1189	0.1115	0.1115
S-ADMM	0.9442	0.8575	0.5248	0.0542

shows the error comparison of the two algorithms at different iteration errors. It is clearly seen that the error of the S-ADMM algorithm is significantly smaller than that of the conventional ADMM algorithm. To show the robustness of the S-ADMM algorithm, we compare the errors of the S-ADMM algorithm and the ADMM algorithm at different $r_p$. According to Figure 2 and

Table 2. Errors under different levels of rank parameter.

${\mathit{r}}_{\mathit{p}}$	0.1	0.15	0.2	0.25	0.3
ADMM	0.0079	0.0183	0.0313	0.0562	0.1115
S-ADMM	0.0027	0.0129	0.0308	0.0435	0.0542

, when the rank of the matrix increases, the trend of increasing error of S-ADMM is always smaller than that of the ADMM algorithm.

5.2. Experiment Results with Real Data

In this section, our algorithmic proposition is substantiated through the empirical validation conducted on two distinct grayscale images sourced from a publicly available dataset. These images exhibit dimensions of $500\times 899$ and $736\times 690$, respectively, thereby providing a diverse testing ground for our methodology. To emulate real-world image corruption scenarios, a deliberate contamination process was introduced. Specifically, 10% of the pixel values were systematically set to 0, while an additional 10% underwent the introduction of random high-level noise. This meticulous contamination scheme was adopted to mimic common noise patterns observed in practical image acquisition and transmission scenarios.

Subsequent to the contamination stage, the denoising process was initiated employing a repertoire of state-of-the-art algorithms, including Singular Value Thresholding (SVT), Alternating Direction Method of Multipliers (ADMM), and the Self-adaptive ADMM (S-ADMM) technique. This multi-faceted approach was undertaken to comprehensively evaluate the efficacy of our algorithm across a spectrum of noise levels and image sizes. Figure 3 represents the first grayscale image after contamination, while Figure 4, Figure 5, Figure 6 and Figure 7 show the denoising effects under different algorithms. Figure 8, Figure 9, Figure 10, Figure 11 and Figure 12 demonstrate the denoising effects under different algorithms for a larger pixel size image. The mean squared error between the denoised image and the ground truth image can be found in

Table 3. MSE and SSIM from different algorithms $(500\times 899)$.

Algorithm	SVT	ADMM	S-ADMM
MSE	0.2503	0.3501	$7.6418\times {10}^{-4}$
SSIM	0.84977	0.74089	0.99395

and

Table 4. MSE and SSIM from different algorithms $(736\times 690)$.

Algorithm	SVT	ADMM	S-ADMM
MSE	0.2737	0.3214	0.0555
SSIM	0.62676	0.5018	0.96393

.

5.3. Experiment Results with Real Noisy Images

In this section, we employed real-world noisy images to evaluate the performance of our proposed algorithm. The data were sourced from a benchmark image denoising database, PolyUDataset [34]. The data were acquired using a Sony A7II ILCE-7M2 camera, manufactured by Sony Corporation in Tokyo, Japan, with an Aperture of f/4, Shutter Speed of 1/200 s, and ISO Value of 3200. The dataset was generated by capturing multiple shots of the same static scene and computing the average image to create the ground truth image for a more accurate assessment of denoising method performance. We selected one frame from these noisy images, with pixel dimensions of $3008\times 1688$.

Figure 13 and Figure 14, respectively, display the ground truth image and the original noisy image. Singular Value Thresholding (SVT), Alternating Direction Method of Multipliers (ADMM), and the Self-adaptive ADMM (S-ADMM) technique were individually applied to process the noisy image, with results shown in Figure 15, Figure 16 and Figure 17. The final mean square error and structural similarity index of the algorithms are presented in

Table 5. MSE and SSIM from different algorithms $(3008\times 1688)$.

Algorithm	SVT	ADMM	S-ADMM
MSE	0.0119	0.1043	0.0093
SSIM	0.80704	0.96089	0.96561

. These results indicate that our proposed S-ADMM method has the best performance.

6. Discussion

The S-ADMM method proposed in this paper, incorporating collaborative sparse regression, has established an effective sparse representation, advancing the exploration of structured sparsity and spatial relationships among pixels. Our adaptive approach has shown remarkable performance in mitigating the impact of penalty parameters on algorithm effectiveness. These findings hold significant implications for the field of image processing, offering insights for enhancing image quality and denoising efficacy. Future research directions could delve into the effectiveness of sparse representation methods, and extend the application of this method to other areas of image processing, thereby driving the development of image processing technologies.

7. Conclusions

In this paper, we proposed a self-adaptive alternating direction method of multipliers for image denoising. By employing collaborative sparse regression, a common sparse representation can be discovered, maintaining structured sparsity within the image and facilitating a more profound exploration of spatial interrelations among pixels. To mitigate the impact of penalty parameters on convergence rates, an adaptive approach based on a balance principle for selecting appropriate penalty parameters is introduced. The experimental results demonstrate that the proposed S-ADMM algorithm yields significantly superior image quality. Other potential enhancements, such as further increasing denoising effectiveness, will be subject to further investigation. Future work could involve integrating deep learning techniques to enhance the performance of the algorithm on complex image data. Additionally, further analysis on multimodal data processing could be conducted to extend the algorithm’s applications to a wider range of scenarios.