*3.1. Data-Driven Redundant Transform*

In this subsection, we first propose a threshold-based reconstruction function, with the assumption that the signal is sparse in the dual frame domain. Then, we present the data-driven redundant transform based on Parseval frames model.

Let {*φ<sup>i</sup>* }*M <sup>i</sup>*=<sup>1</sup> be a frame and {*ψ<sup>i</sup>* }*M <sup>i</sup>*=<sup>1</sup> be its dual frame. For convenience, we stack them as the matrices **Φ** = [*φ*1, *φ*2, ... , *φM*] and **Ψ** = [*ψ*1, *ψ*2, ... , *ψN*], respectively. Let **x** = **x**ˆ + **e** be a signal vector, where **x**ˆ is the original noiseless signal and **e** is a zero-mean white Gaussian noise. The frame reconstruction function (6) can be formulated as **x** = **ΦΨ***T***x** = **ΦΨ***T*(**x**ˆ + **e**). By assuming the sparse prior of signals over the **Ψ** domain, we apply a columnwise hard thresholding operator S*λ*(·) (which shall be defined in the next subsection) on **Ψ***T*(**x**ˆ + **e**), such that

$$\hat{\mathbf{x}} = \boldsymbol{\Phi} \mathcal{S}\_{\lambda} (\boldsymbol{\Psi}^T \mathbf{x}),\tag{7}$$

where *λ* is a vector with elements *λ<sup>i</sup>* corresponding to *ψ<sup>i</sup>* , *<sup>i</sup>* = 1, 2, ... , *<sup>M</sup>*. Apparently, S*λ*(**Ψ***T***x**) is the sparse coefficients of **x** under **Ψ** in the sense of an analysis model, while it also serves as the sparse coefficients under **Φ** in the sense of a synthesis model. In other words, Equation (7) admits that the synthesis and analysis models share almost the same sparse coefficients.

As we all know, the standard orthogonal basis, which is a significant tool in signal representation and transformation, is a special kind of frame with frame bounds *A* = *B* = 1. In fact, the standard orthogonal basis is a special case of a Parseval frame. In order to exceed the so-called *perfect reconstruction property* of the standard orthogonal basis in signal representation and transform, we refer to the Parseval frame. Therefore, we propose the data-driven redundant transform based on Parseval frame (DRTPF), as follows

$$\mathbf{y} \leftarrow \mathcal{S}\_{\lambda}(\boldsymbol{\Psi}^{\mathrm{T}}\mathbf{x}),\tag{8}$$

$$
\hat{\mathbf{x}} \leftarrow \Phi \mathbf{y},
\tag{9}
$$

$$\text{s.t. } \Psi \Phi = \mathbf{I}, \tag{10}$$

$$\|\mathbf{y}\|\_{0} \le s\_{\prime}$$

$$\sum\_{i=1}^{M} s\_{\prime} \qquad \ldots \qquad s\_{\prime} \qquad \ldots \qquad \ldots \qquad \ldots \qquad \ldots \qquad \ldots \qquad \ldots$$

$$\sum\_{i=1} |<\mathbf{x}, \boldsymbol{\Phi}\_i >|^2 = \|\mathbf{x}\|\_{2'}^2 \tag{11}$$

where (8) is the forward transform and (9) is the backward transform. The relationship between **Φ** and **Ψ** is formulated as (10), which implies the relationship between the frame and its dual frame. The forward transform operator **Ψ** is also a Parseval frame, as it is a dual frame of **Φ**. Thus, the projection of the signal **x** over the **Ψ** domain can be formulated as

$$\sum\_{i=1}^{M} |<\mathbf{x}, \Psi\_i>|^2 = \|\mathbf{x}\|\_2^2. \tag{12}$$

Equation (12) indicates that the transform coefficients of the proposed DRTBF are bounded by the original signal **x**. This constraint leads to a more robust result than traditional sparse models.

To convert DRTPF into an optimization problem, (11) can be written as the more compact expression **ΦΦ***<sup>T</sup>* = **I**, which characterizes **Φ** in a way that is unrelated to the data. This property indicates that the rows of the frame **Φ** are orthogonal, thus satisfying the so-called *perfect reconstruction property* which ensures that a given signal can be perfectly represented by its canonical expansion (in a manner similar to orthogonal bases).

Assuming **<sup>X</sup>** <sup>∈</sup> <sup>R</sup>*N*×*<sup>L</sup>* is the training data with signal vectors **<sup>x</sup>***<sup>i</sup>* <sup>∈</sup> <sup>R</sup>*N*, *<sup>i</sup>* <sup>=</sup> 1, 2, ... , *<sup>L</sup>* as its columns, an optimization model for training DRTPF can be written as

$$\min\_{\boldsymbol{\Phi}, \boldsymbol{\Psi}, \boldsymbol{\lambda}, \boldsymbol{Y}} \left\lVert \boldsymbol{\lambda} - \boldsymbol{\Phi} \mathbf{Y} \right\rVert\_{F}^{2} + \eta\_{1} \left\lVert \mathbf{Y} - \mathcal{S}\_{\lambda} \left(\mathbf{Y}^{T} \mathbf{x}\right) \right\rVert\_{F}^{2} + \eta\_{2} \left\lVert \mathbf{Y} \right\rVert \left\lVert \boldsymbol{0} + \eta\_{3} \left\lVert \boldsymbol{\Phi} \mathbf{Y}^{T} - \mathbf{I} \right\rVert\_{F}^{2}$$
 
$$\text{s.t. } \boldsymbol{\Phi} \boldsymbol{\Phi}^{T} = \mathbf{I}. \tag{13}$$

The dual frame condition **ΦΨ***<sup>T</sup>* = **I** and the Parseval frame condition **ΦΦ***<sup>T</sup>* = **I** imply that the difference of **Φ** and **Ψ** is in the null space of **Φ**. Denote [**a***<sup>T</sup>* <sup>1</sup> , **<sup>a</sup>***<sup>T</sup>* <sup>2</sup> , ··· , **<sup>a</sup>***<sup>T</sup> N*] *<sup>T</sup>* = **<sup>Φ</sup>** − **<sup>Ψ</sup>**. The vectors **a***i*, *i* = 1, 2, ··· , *N* are orthogonal to **Φ**. Thus, it is clear that the dual frame **Ψ** contains two subspaces: one spanned by **Φ** and the one spanned by the **a***i*, *i* = 1, 2, ··· , *N*.
