**Frame Update Step:**

2: Update **<sup>Ψ</sup>** in column-wise. Compute **<sup>W</sup>** = S(**Ψ***T***X**).

For *i* = 1 : *M*

Denote <sup>J</sup><sup>ˆ</sup> *<sup>i</sup>* as the indices of zeros in the *i*-th column of *W*. Set *ψ<sup>T</sup> <sup>i</sup>* **<sup>X</sup>**J<sup>ˆ</sup> *<sup>i</sup>* = 0. Compute *ψ<sup>i</sup>* via Equation (34).

End For

3: Update **Φ** via Gradient Descent and singular value map.

For k=1:*r*

Calculate Gradient via Equation (39) and calculate the descent step via Equation (40)

End for

4: Apply SVD decomposition **Φ** = **UΣV***<sup>T</sup>* , map **Σ** to obtain **Σ**ˆ and reconstruct **Φ** = **UΣ**ˆ **V***T*.

**End for**

#### **5. Restoration**

The image restoration aims to reconstruct a high-quality image I from its degraded (e.g., noisy, blurred and/or downsampled) version L, denoted by L = *SH*I + **n**, where *H* represents a blurring filter, *S* the downsampling operator, and **n** is a noisy signal. For the signal satisfies the SSM-NTF, the restoration model based on SSM-NTF is formulated as

$$\{\mathcal{Z}, \hat{\mathbf{Y}}, \hat{\boldsymbol{\lambda}}\} = \min\_{\mathcal{Z}, \{\mathbf{y}\_i\}\_{i=1}^N, \hat{\boldsymbol{\lambda}}} \|\mathcal{L} - S\mathcal{H}\mathcal{Z}\|\_F^2 + \gamma \sum\_i \|\mathbf{R}\_i \mathcal{Z} - \boldsymbol{\Phi}\mathbf{y}\_i\|\_F^2 + \gamma\_1 \sum\_i \|\mathbf{y}\_i - \mathcal{S}\_{\lambda}(\mathbf{Y}^T \mathbf{R}\_i \mathcal{Z})\|\_F^2 + \gamma\_2 \sum\_i \|\mathbf{y}\_i\|\_0 \tag{41}$$

where **R***<sup>i</sup>* is an operator that extracts the *i*-th patch of the image I and **y***<sup>i</sup>* is the *i*-th column of **Y** . *<sup>λ</sup>* denotes a vector [*λ*1, *<sup>λ</sup>*2, ··· , *<sup>λ</sup>M*] with *<sup>λ</sup><sup>j</sup>* operating on the *<sup>j</sup>*-th element of **<sup>Ψ</sup>***T***R***i*I. On the right side of Equation (41), the first term is the global force that demands the proximity between the degraded image L, and its high-quality version I. The rest terms are the local constraints to make sure every patch at location *i* satisfies the SSM-NTF.

To solve Problem (41), we apply Algorithm 1 to obtain the sparse coefficients **Y** and the threshold values *<sup>λ</sup>*. We mainly state the iterative method to obtain <sup>I</sup>. Assume the sign of **<sup>Ψ</sup>***T***R***i*I*<sup>k</sup>* will not change much between two steps, we set it in the *<sup>k</sup>*-th step by **<sup>c</sup>***<sup>k</sup>* <sup>=</sup> *sign*(**Ψ***T***R***i*I*k*−1). where *sign* is the sign function. Denote **<sup>d</sup>***<sup>k</sup>* <sup>=</sup> **<sup>Ψ</sup>***T***R***i*I*k*−1. We set **<sup>O</sup>***<sup>k</sup>* as an index set that satisfies <sup>|</sup>**d***l*| ≤ *<sup>λ</sup>l*, *<sup>l</sup>* <sup>∈</sup> **<sup>O</sup>***k*. Set **<sup>u</sup>***<sup>k</sup>* ∈ R*<sup>M</sup>* as a vector with elements **<sup>u</sup>***<sup>l</sup>* = - *<sup>λ</sup><sup>l</sup> <sup>l</sup>* ∈ **<sup>O</sup>***k*, 0 *otherwise*. Then the non-convex and non-smooth

threshold can be removed with the substitution that **<sup>y</sup>***<sup>i</sup>* − S*λ*(**Ψ***T***R***i*I*k*) <sup>≈</sup> **<sup>y</sup>***<sup>i</sup>* <sup>+</sup> **<sup>c</sup>***<sup>k</sup>* **<sup>u</sup>** <sup>−</sup> **<sup>Ψ</sup>***T***R***i*I*<sup>k</sup>* Thus, in the *k*-th step, the problem needs to be solved is expressed as

$$\{\hat{\mathcal{L}}^k\} = \min\_{\mathcal{Z}^{k-1}} \|\mathcal{L} - SH\mathcal{Z}^{k-1}\|\_F^2 + \gamma \sum\_i \|\mathbf{R}\_i \mathcal{Z}^{k-1} - \Phi \mathbf{y}\_i\|\_F^2 + \gamma\_1 \sum\_i \|\mathbf{y}\_i + \mathbf{c}^k \odot \mathbf{u}^k - \mathbf{Y}^T \mathbf{R}\_i \mathcal{Z}^{k-1}\|\_F^2 \tag{42}$$

where is point multiplication. This convex problem can be easily solved by gradient descent algorithm. We summarized the restoration algorithm in Algorithm 3.

**Algorithm 3** Restoration algorithm
