**1. Introduction**

Sparse representation of signals in dictionary domains has been widely studied and has provided promising performance in numerous signal processing tasks such as image denoising [1–5], super resolution [6–8], inpainting [9,10] and compression [11,12]. It is well known that images are represented by a linear combination of certain atoms of a dictionary. Overcomplete sparse representation is the overcomplete system with a sparse constraint. Common overcomplete systems differ from the traditional bases, such as DCT, DFT and Wavelet, because they offer a wider range of generating elements; potentially, this wider range allows more flexibility and effectiveness in signal sparse representation. However, it is a severely under-constrained illposed problem to find the underlying overcomplete representation due to the redundancy of the systems. When the underlying representation is sparse and the overcomplete systems have stable properties, the ill-posedness will disappear [13]. Sparse models are generally classified into two categories: Synthesis sparse models and analysis sparse model [14]. The commonly referred to sparse models are synthesis sparse models. The analysis ones characterize the signal by multiplying it with an analysis overcomplete dictionary, leading to a sparse outcome. A variety of effective sparse models have been investigated and established such as the classical synthesis sparse model [9,15], the classical analysis sparse model [14], the nonlocal sparse model [16,17] and the 2D sparse model [18]. Unfortunately, these models ignore the stability recovery property which claims that once a sufficient sparse solution is found, all alternative solutions

necessarily reside very close to it [9]. Recently, the stable recovery of sparse representation has drawn attention in signal processing theory. Generally speaking, stable recovery can be guaranteed by two properties: Sufficient sparsity and a favorable structure of the dictionary [19]. Donoho defines the concept of mutual incoherence of the dictionary and applies it to prove some possibility of stable recovery [19]. The authors of [20] proposea sparsity-based orthogonal dictionary learning method to minimize the mutual incoherence. The authors of [21] propose an incoherent dictionary learning scheme by integrating a low rank gram matrix of the dictionary into the dictionary learning model.

A more powerful stable recovery guarantee developed by Candes and Tao, termed Restricted Isometry Property (RIP), makes consequent analysis easy [22]. A matrix **Φ** is said to satisfy the RIP of order *k* if there exists a constant *δ<sup>k</sup>* ∈ (0, 1) such that

$$(1 - \delta\_k) \|\mathbf{y}\|\_2^2 \le \|\Phi \mathbf{y}\|\_2^2 \le (1 + \delta\_k) \|\mathbf{y}\|\_2^2 \tag{1}$$

holds for all *k*-sparse vectors **y**. *δ<sup>k</sup>* is defined as the smallest constant which satisfies the above inequalities and is called the restricted isometry constant of **Φ**.

Most RIP research substantially investigates applying RIP as a stablility analysis instrument [17,23,24] or finding optimal RIP constant [25,26] which are all theoretical analyses rather than practical applications. According to the research of [21], the intrinsic property of a dictionary has a direct influence on its performance. All familiar algorithms are staggeringly unstable with a coherent or degenerate dictionary [19]. Recognizing the gap between theoretical analyses and practical applications of RIP, this paper aims to build a stable sparse model satisfying the RIP.

Recently, the frame as a stable overcomplete system has drawn some attention in signal processing as the given signal can be represented by its canonical expansion in a manner similar to conventional bases under the frame. Some data-driven approaches are proposed in [1,27–30]. The authors of [27,29,30] utilize redundant tight frame in compressed sensing and [28] applies tight frame to few-view image reconstruction. Study [1] presents a data-driven method that the dictionary atoms associated with the tight frame are generated by filters. These approaches achieve much better image processing performance than previous methods, and meanwhile the tight frame condition which requires the frame almost-orthogonality will limit the flexibility in sparse representation. Study [31] derives stable recovery result for *l*1-analysis minimization in redundant, possibly non-tight frames. Inspired by this result and the relationship between RIP and frame, we aim to establish a stable sparse model with RIP based on non-tight frame.

We call a sequence {*φ<sup>i</sup>* }*M <sup>i</sup>*=<sup>1</sup> ∈ **H** a frame if and only if there exist two positive numbers *A* and *B* such that

$$\|A\|\|\mathbf{x}\|\|\_{2}^{2} \le \sum\_{i=1}^{M} |<\mathbf{x}, \Phi\_{i}>\rangle^{2} \le \|B\|\|\mathbf{x}\|\|\_{2}^{2} \quad \forall \mathbf{x} \in \mathbf{H}^{N} \tag{2}$$

Here, *A* and *B* are called the bounds of the frame. We find that every submatrix **Φ***<sup>k</sup>* satisfied RIP is a non-tight frame with (1 − *δk*) and (1 + *δk*) as its frame bounds with a given *k*. Obviously, there is an essential connection between the non-tight frame and the RIP.

In this paper we focus on a stable sparse model and more specifically on the development of an algorithm that would learn a pair of non-tight frame based dictionaries from a set of signal examples. We propose a stable sparse model via applying the non-tight frame condition to approximate the RIP. This model shares the favorite overcomplete structures with the common sparse models, and meanwhile it contains RIP and closed-form sparse coefficient expression which ensure stable recovery. Recognizing that the optimal framebounds are essentially the maximum and minimum singular values of the frame, RIP is actually enforced on the dictionary pair (the frame and its dual frame) by constraining the singular values of them. We also formulate a dictionary pair learning model via applying the second-order truncated Taylor series to approximate the inverse frame operator. Then we present an efficient algorithm to learn the dictionary pair via a two-phase iterative approach. To summarize, this paper makes the following contributions:


This paper is organized as follows: Section 2 reviews the related work on frame, synthesis sparse model and analysis sparse model. Section 3 presents our stable sparse model with non-tight frame SSM-NTF along with a dictionary pair learning model. Section 4 proposes the corresponding dictionary pair learning algorithm. Section 5 proposes the image restoration method of our proposed SSM-NTF model. In Section 6 we analyze the computational complexity of our proposed algorithm. In Section 7, we demonstrate the the effectiveness of our SSM-NTF model by analyzing the convergence of the corresponding algorithm, denoising natural and piecewise constant images, super resolution and image inpainting. Finally, Section 8 concludes this paper.

### **2. Related Work**

In this section, we briefly review the related work on frame, synthesis sparse model and analysis sparse model.

Frame: A frame **Φ**is called a tight frame if the frame bounds are equal in the Equation (2) [32]. There are two associated operators can be defined between the Hilbert space **H***<sup>N</sup>* and Square Integrable Space **l***<sup>M</sup>* <sup>2</sup> (·) once a frame is defined. One is the analysis operator **T** defined by

$$(\mathbf{T}\mathbf{x})\_i = <\mathbf{x}, \boldsymbol{\Phi}\_i >\_{\prime} \quad \forall \mathbf{x} \in \mathbf{H}^N \tag{3}$$

and the other is its adjoint operator **T**∗ which is called the synthesis operator

$$\mathbf{T}^\*\mathbf{c} = \sum\_{i=1}^{M} \mathbf{c}\boldsymbol{\phi}\_i \quad \forall \mathbf{c} = (\mathbf{c}\_i)\_{i \in f} \in \mathbf{l}\_2^M(\mathbf{T}) \tag{4}$$

then, the frame operator can be defined as the following canonical expansion

$$\mathbf{Fx} = \mathbf{T}^\* \mathbf{T} \mathbf{x} = \sum\_{i=1}^{M} < \mathbf{x}, \boldsymbol{\phi}\_i > \boldsymbol{\phi}\_i \tag{5}$$

In Euclidean space, a given frame Φ can be represent in manner of matrix with its columns of it as the frame elements. Then one of its adjoint operator can be representated as *<sup>ψ</sup><sup>i</sup>* = **<sup>F</sup>**−1*φ<sup>i</sup>* [32]. Let**<sup>x</sup>** <sup>∈</sup> <sup>R</sup>*<sup>N</sup>* be an arbitrary vector, a reconstruction function can be expressed as the following form

$$\mathbf{x} = \sum\_{i=1}^{M} < \mathbf{x}, \boldsymbol{\psi}\_{i} > \boldsymbol{\phi}\_{i} \tag{6}$$

Synthesis sparse model: The conventional synthesis sparse model represents a vector **x** by the linear combination of a few atoms from a large dictionary **Φ**, denoted as **x** = **Φy**, **y**<sup>0</sup> ≤ *L*, where *L* is the sparsity of **y**. The computational techniques for approximating sparse coefficient **y** under a given dictionary **Φ** and **x** includes greedy pursuit (e.g., OMP [9]) and convex relaxation optimization, such as Lasso [33] and FISTA [8]. In order to improve the performance of sparse representation, some modified models such as the nonlocal sparse model [16], the frame based sparse model [21], and the MD sparse model [18] are also investigated.

Analysis sparse model: The analysis sparse model is defined as: **y** = **Ωx**, **y**<sup>0</sup> = *p* − *l* where **Ω** ∈ R*p*×*<sup>d</sup>* is a linear operator (also called as a dictionary), and *<sup>l</sup>* denotes the co-sparsity of the signal **<sup>x</sup>**. The analysis representative vector **y** is sparse with *l* zeros. The zeros in **y** denote the low-dimensional subspace to which the signal **x** belongs. The analysis sparse coding [14] and dictionary learning [34] approach are also been proposed.

However, all these models ignore the stability recovery property which provides stable reconstruction of the signals in presence of noise.

Dictionary learning methods: The dictionaries include analytical dictionaries, such as DCT, DWT, curvelets and contourlets and learned dictionaries. Some dictionary learning method are proposed, such as the classical KSVD [9] algorithm, the efficient sparse coding which convert the original dictionary learning problem to two least squares problem by applying the Lagrange dual [3], the non-local sparse model [16] which learns a set of PCA sub-dictionaries by cluster the samples into K clusters using image nonlocal self-similarity prior and its improved version which using the *lq*-norm to instead the *l*2-norm in order to handle different image contents. With the realization of stability, some mutual-coherence based methods are proposed. In [20] a sparsity-based orthogonal dictionary learning method is proposed to minimize the mutual-coherence of the dictionary. The authors of [21] propose an in coherent dictionary learning scheme by integrating a low rank gram matrix of the dictionary into the dictionary learning model. However, these methods only concern the capability of the dictionary without modeling the sparse coefficients which still has some probability of instability.
