**3. The Proposed SSM-NTF**

In this section, we present the stable sparse model with non-tight frame, (Section 3.1), the stability analysis of the proposed model, (Section 3.2) and the dictionary pair (the frame pair) learning model, (Section 3.3).

### *3.1. Stable Sparse Model with Non-Tight Frame*

In this section, we derive our stable sparse model with non-tight frame where the non-tight frame condition serves as an approximation to the RIP.

According to [35], a *k*-th RIP constant can be express as

$$\delta\_k(\Phi) = \frac{\Gamma\_k(\Phi) - 1}{\Gamma\_k(\Phi) + 1} \tag{7}$$

where

$$
\Gamma\_k(\Phi) = \frac{\theta\_{\text{max}}^k}{\theta\_{\text{min}}^k} \tag{8}
$$

$$\theta\_{\max}^k = \max\_{\|\mathbf{y}\|\_0 = k} \frac{\|\Phi \mathbf{y}\|\_2^2}{\|\mathbf{y}\|\_2^2}, \theta\_{\min}^k = \min\_{\|\mathbf{y}\|\_0 = k} \frac{\|\Phi \mathbf{y}\|\_2^2}{\|\mathbf{y}\|\_2^2} \tag{9}$$

The Equation (7) provides a new perspective in integrating the RIP to sparse model via applying *θk max* and *θ<sup>k</sup> min* instead of the RIP constant *δk*(**Φ**). The difficulty in building a stable sparse model decreases. However, the sparsity *k* varies with the noise level, and also, in a feasible numerical calculation method, it is impossible to sweep through all the samples satisfying **x**<sup>0</sup> = *k* to pursue an unknown dictionary **Φ**.

Let **x** be a signal vector, the frame reconstruction function can be formulated as **x** = **ΦΨ***T***x** where **Ψ** is a dual frame of **Φ**. Adding a reasonable sparsity prior to the signal **x** over **Ψ** domain, we can derive

$$\frac{\|\|\Phi(\mathbf{Y}^T\mathbf{x})\|\|\_2^2}{\|\|\mathbf{Y}^T\mathbf{x}\|\|\_2^2} = \frac{\|\|\mathbf{x}\|\|\_2^2}{\|\|\mathbf{Y}^T\mathbf{x}\|\|\_2^2} = \frac{1}{\frac{\|\|\mathbf{Y}^T\mathbf{x}\|\|\_2^2}{\|\|\mathbf{x}\|\|\_2^2}}\tag{10}$$

Denoting the optimal frame bounds of **Φ** as *A* and *B*, the frame condition of **Ψ** can be formulated as <sup>1</sup> *<sup>B</sup>* <sup>≤</sup> **<sup>Ψ</sup>***T***<sup>x</sup>**<sup>2</sup> 2 **<sup>x</sup>**<sup>2</sup> 2 <sup>≤</sup> <sup>1</sup> *<sup>A</sup>* . Then a pair of bounds for Equation (10) can be obtained as *<sup>A</sup>* <sup>≤</sup> **<sup>Φ</sup>**(**Ψ***T***x**)<sup>2</sup> 2 **<sup>Ψ</sup>***T***<sup>x</sup>**<sup>2</sup> 2 ≤ *B*. A formula similar to Equation (9) is derived as *B* = max **x**∈J **<sup>Φ</sup>**(**Ψ***T***x**)<sup>2</sup> 2 **<sup>Ψ</sup>***T***<sup>x</sup>**<sup>2</sup> 2 , *A* = min **x**∈J **<sup>Φ</sup>**(**Ψ***T***x**)<sup>2</sup> 2 **<sup>Ψ</sup>***T***<sup>x</sup>**<sup>2</sup> 2 where J is the data set. Imitating Equation (7), we can obtain a RIP-like constant expression

$$\hat{\delta}(\Phi) = \frac{\hat{\Gamma}(\Phi) - 1}{\hat{\Gamma}(\Phi) + 1} \tag{11}$$

where Γˆ(**Φ**) = *<sup>B</sup> <sup>A</sup>* . Obviously, <sup>ˆ</sup> *δ*(**Φ**) can be regarded as an approximation of the RIP constant which benefits the computation due to the ignorance on sparsity degree. In a word, the RIP constraint can be satisfied by constraining the frame bounds. Thus, a stable overcomplete system with a sparsity prior can be established.

Now we discuss the characteristic of the frame bounds *A* and *B*. The Frame Condition (2) has a more compact form <sup>√</sup>*<sup>A</sup>* <sup>≤</sup> *<sup>η</sup>***<sup>Φ</sup>** <sup>≤</sup> <sup>√</sup>*<sup>B</sup>* where *<sup>η</sup>***<sup>Φ</sup>** denotes any singular value of **<sup>Φ</sup>**. More specifically, <sup>√</sup>*<sup>A</sup>* <sup>=</sup> *<sup>η</sup>min*, <sup>√</sup>*<sup>B</sup>* <sup>=</sup> *<sup>η</sup>max* where *<sup>η</sup>max* and *<sup>η</sup>min* denote the maximum and minimum singular values of **<sup>Φ</sup>**, respectively. Then, we can obtain *<sup>η</sup>max* ≥ *<sup>θ</sup><sup>k</sup> max*, *<sup>η</sup>min* ≤ *<sup>θ</sup><sup>k</sup> min*. It is easy to know that <sup>ˆ</sup> *δ*(**Φ**) ≥ *δk*(**Φ**). Obviously, ˆ *δ*(**Φ**) is a reasonable relaxation of *δk*(**Φ**) as ˆ *δ*(**Φ**) is slightly exceed *δk*(**Φ**) but resides very close to it as long as the data is not seriously degraded. Therefore, the RIP constraint can be enforced on the frames by limiting the maximum and minimum singular values.

In this paper, we integrate non-frame to traditional sparse model to establish a stable sparse model with RIP. Let **x** be a signal vector. Under the assumption of the sparsity prior of **Ψ***T***x**, we apply a soft thresholding operator S*λ*(·) (which shall be defined in the next subsection) on it such that

$$\mathbf{x} = \boldsymbol{\Phi} \mathcal{S}\_{\lambda}(\mathbf{Y}^T \mathbf{x}) \tag{12}$$

where *λ* is a vector with elements *λ<sup>i</sup>* corresponding to *ψ<sup>i</sup>* , *i* = 1, 2, ... , *M*. Therefore, we propose the stable sparse model with non-tight frame (SSM-NTF) as follows

$$\mathbf{y} = \mathcal{S}\_{\lambda}(\mathbf{Y}^{T}\mathbf{x}), \quad \mathbf{x} = \boldsymbol{\Phi}\mathbf{y}, \tag{13}$$
 
$$\text{s.t. } \|\mathbf{y}\|\_{0} \le s.$$

Here, the correlation between the frame **Φ** and its dual frame **Ψ** is formulated as **Ψ** = **F**−1**Φ**. The frame operator **F** is formulated as **ΦΦ***<sup>T</sup>* which is indeed a gram matrix of **Φ**. The singular values of **<sup>Φ</sup>** are constrained by <sup>√</sup>*<sup>A</sup>* <sup>≤</sup> *<sup>η</sup>***<sup>Φ</sup>** <sup>≤</sup> <sup>√</sup>*<sup>B</sup>* to satisfy the RIP. Actually, by constraining the singular values of **Φ**, the elements of the gram matrix are also bounded which meets the theory of mutual coherence.

In order to be consistent with the traditional sparse models, we refer to the frame **Φ** and its dual frame **Ψ** as dictionary and its dual dictionary.

#### *3.2. The Stability Analysis of the Proposed Model*

In sparse representation problem, a given noiseless signal **x**, can be formulated formulated as

$$(P\_0): \quad \min\_{\mathbf{x}} \|\mathbf{y}\|\_0 \quad \text{s.t.} \,\mathbf{x} = \Phi \mathbf{y} \tag{14}$$

where **Φ** is the sparse representation dictionary and **y** is the sparse coefficients. While **x** = **Φy** is an underdetermined linear system, the problem (*P*0) has the unique solution **y**<sup>0</sup> as soon as it satisfies the uniqueness property which is formulated as

$$\|\|\mathbf{\hat{y}}\|\|\_{0} < \frac{1}{2} (1 + 1/\mu) \tag{15}$$

where μ is the mutual-coherence of **Φ** [9]. However, the signals are usually acquired with noise, then the problem (*P*0) should be relaxed to the problem (*P* ) which is expressed as

$$\|(P\_{\mathfrak{e}}):\quad\min\_{\mathbf{x}}\|\|\mathbf{y}\|\|\_{0}\quad\text{s.t.}\;\|\mathbf{x}-\Phi\mathbf{y}\|\|\_{F}^{2}\leq\mathfrak{e}\tag{16}$$

where  is an error-tolerant which exists due to the noise. The problem (*P* ) will no longer maintain the uniqueness of solution as **x** = **Φy** +  is an inequality system. Thus, the notion of Uniqueness Property (15) is replaced by the notion of stability which claims that all the alternative solutions reside very close to the ideal solution. Under the stable guarantee, we can yet ensure that the recovery results of our methods produce meaningful solutions. Assume that **y**<sup>0</sup> is the ideal solution to the problem (*P* ) and ˆ**y** is the candidate one, the traditional sparse model has a stability claim of the form [9]

$$\|\mathbf{\hat{y}} - \mathbf{y}\_0\|\_2^2 \le \frac{4\epsilon^2}{1 - (2s\_0 - 1)\mu'} \tag{17}$$

where μ is the mutual coherence which is formulated as *μ* = max*i*=*<sup>j</sup>* | < *φ<sup>i</sup>* , *φ<sup>j</sup>* > |, *i*, *j* = 1, 2, ··· , *M*. Apparently, the error bound of Equation (17) can only be determined with given sparsity *s*<sup>0</sup> and the mutual coherence μ. However, the mutual coherence of an unknown dictionary is very difficult to calculate which lead to a result that we can not ensure the stability in the dictionary learning case. In contrast, we derive a similar stability claim of our proposed SSM-NTF model.

Defining **d** = **y**ˆ − **y**<sup>0</sup> with **y**<sup>0</sup> as the ideal solution to the model, we have that **Φy**ˆ − **Φy**<sup>0</sup><sup>2</sup> = **Φd**<sup>2</sup> ≤ 2. From the previous subsection, we have know that the frame **Φ** satisfies the RIP with the corresponding parameter ˆ *δ*(**Φ**). Thus, using this property and exploiting the lower-bound part in Equation (1), we get

$$(1 - \delta(\Phi)) \|\mathbf{d}\|\_2^2 \le \|\Phi \mathbf{d}\|\_2^2 \le 4\epsilon^2 \tag{18}$$

where ˆ *<sup>δ</sup>*(**Φ**) = <sup>Γ</sup>ˆ(**Φ**)−<sup>1</sup> <sup>Γ</sup>ˆ(**Φ**)+<sup>1</sup> <sup>=</sup> *<sup>B</sup> <sup>A</sup>* −1 *B <sup>A</sup>* +1 . Thus, we get a stability claim of the form

$$\|\|\mathbf{d}\|\|\_{2}^{2} = \|\mathbf{\hat{y}} - \mathbf{y}\_{0}\|\|\_{2}^{2} \le \frac{4\epsilon^{2}}{1 - \frac{\frac{\mathcal{E}}{\mathcal{E}} - 1}{\frac{\mathcal{E}}{\mathcal{X}} + 1}}\tag{19}$$

Obviously, the error bound of the SSM-NTF is determined by *<sup>B</sup> <sup>A</sup>* , the ratio of the upper bound to the lower bound of the frame, rather than the specific values of *A* and *B*. Thus, for the convenience of numerical experiments, we usually set *A* to a fixed value. A main advantage of standard orthogonal transformations is that they maintain the energy of the signals in the transform domain as its frame bounds *A* and *B* are equal to 1. However, the standard orthogonal basis is non-redundant that limits its performance in sparse representation. In order to make a trade off between the represent accuracy and the degree of redundant, we usually set the lower frame bound *A* to a value a little smaller than 1 but not over-small as *A* is the minimum singular value of **Φ** which determines the condition number of **Φ**. Thus, once the tolerance error is given, the value of *B* can be easily calculated. Further, a pair of dictionaries conform to the given error can be obtained using the proposed SSM-NTF model. On the other hand, if the value of *B* is given by experience, the error bound of our model can be measured.
