Wavelet Thresholding Risk Estimate for the Model with Random Samples and Correlated Noise

Abstract

Signal de-noising methods based on threshold processing of wavelet decomposition coefficients have become popular due to their simplicity, speed, and ability to adapt to signal functions with spatially inhomogeneous smoothness. The analysis of the errors of these methods is an important practical task, since it makes it possible to evaluate the quality of both methods and equipment used for processing. Sometimes the nature of the signal is such that its samples are recorded at random times. If the sample points form a variational series based on a sample from the uniform distribution on the data registration interval, then the use of the standard threshold processing procedure is adequate. The paper considers a model of a signal that is registered at random times and contains noise with long-term dependence. The asymptotic normality and strong consistency properties of the mean-square thresholding risk estimator are proved. The obtained results make it possible to construct asymptotic confidence intervals for threshold processing errors using only the observed data.

1. Introduction

In digital signal processing tasks, it is often assumed that the recorded signal samples are independent. However, there are many physical processes that demonstrate long-term dependence where correlations between observations decrease rather slowly. For example, long-term dependence is often observed in geophysical processes where it takes the form of long periods of large or small values of observations. Interferences in communication channels demonstrate similar phenomena. Wavelet methods are widely used in the analysis and processing of signals recorded during the study of such processes.

The wavelet decomposition of a function $f\left(x\right)$ is a series

$$f\left(x\right)=\sum _{j,k\in Z}\langle f,{\psi }_{j,k}\rangle {\psi }_{j,k}\left(x\right),$$

where ${\psi }_{j,k}\left(x\right)=2^{j/2}\psi (2^{j}x-k)$, and $\psi \left(x\right)$ is a wavelet function. The indices j and k are called the scale and the shift, respectively. This decomposition provides a time/scale representation of the signal function, that allows one to localise its features. There exist many wavelet functions with various properties.

In practice, a discrete wavelet transform is used. It is a multiplication of a sampled signal vector by an orthogonal matrix defined by the wavelet function $\psi \left(x\right)$ (in practice implemented with a fast cascade algorithm [1]). This transform is applied to data, and the threshold processing of the resulting wavelet coefficients is performed [1]. For a model of signal samples with an equispaced grid, these methods were well studied by D. Donoho, I. Johnstone, B. Silverman and others [2,3,4,5,6,7,8,9,10]. Statistical properties of the mean-square risk estimator were also studied. It is shown that under certain conditions it is strongly consistent and asymptotically normal [11,12,13].

In some experiments it is not possible to record signal samples at regular intervals [14]. Sometimes registration of samples is made at random times. It was shown by T. Cai and L. Brown [15] that, if the sample points form a variational series based on a sample from the uniform distribution on the data registration interval, then the rate of the mean-square thresholding risk remains, up to a logarithmic factor, equal to the optimal rate in the class of Lipschitz regular functions. A special case of the uniform distribution appears, for example, when considering a Poisson process, and since the conditional distribution of its points on a given time interval, given the number of points, is uniform. These models can arise, for example, in astronomy when considering the stellar intensity. In this paper, it is proven that under some regularity conditions, the statistical properties of the risk estimator also remain the same for both equispaced and random sample grids.

2. Long-Term Dependence

Let the signal function $f\left(x\right)$ be defined on the segment $[0,1]$ and be uniformly Lipschitz regular with some exponent $\gamma >0$ and Lipschitz constant $L>0$: $f\in \mathrm{Lip}(\gamma ,L)$. Assume that the samples of $f\left(x\right)$ contain additive correlated noise and are recorded at random times that are independent and uniformly distributed on $[0,1]$. Namely, consider the following data model:

$$Y_i=f\left(x_{\left(i\right)}\right)+e_i,i=1,\dots ,N(N=2^J),$$

(1)

where $0\le x_{\left(1\right)}<\dots <x_{\left(N\right)}\le 1$ is the variational series based on a sample from the uniform distribution on the segment $[0,1]$, and $\{e_i,i\in \mathbb{Z}\}$ is a stationary Gaussian process with the covariance sequence $r_k=\mathbf{cov}(e_i,e_{i+k})$. We assume that $e_i$ have zero mean and unit variance. We also assume that the noise autocovariance function decreases at the rate of $r_k\sim k^{-\alpha }$, where $0<\alpha <1$. This corresponds to the long-term dependence between observations [7].

The observations consist of pairs $(x_{\left(1\right)},Y_1),\dots ,(x_{\left(N\right)},Y_N)$, where the distances between the samples are, generally, not equal. It is known that $\mathsf{E}{x}_{\left(i\right)}=i/(N+1)$ (see Lemma 2 in [15]). Along with (1), consider a sample with equal distances between sample points

$$\left(\frac{1}{N+1},Z_1\right),\dots ,\left(\frac{N}{N+1},Z_N\right).$$

(2)

where

$$Z_i=f\left(\frac{i}{N+1}\right)+e_i,i=1,\dots ,N.$$

For the sample (2) threshold processing methods have been developed that effectively suppress the noise and provide an “almost” optimal rate of the mean-square risk [7,8]. The discrete wavelet transform with Meyer wavelets is applied to the sample (2) to obtain a set of empirical wavelet coefficients [1]

$$W_{j,k}={\mu }_{j,k}+2^{\frac{(1-\alpha )(J-j)}{2}}{\xi }_{j,k},j=0,\dots ,J-1,k=0,\dots ,2^j-1,$$

where ${\mu }_{j,k}$ are discrete wavelet coefficients of the sample

$$f\left(\frac{1}{N+1}\right),\dots ,f\left(\frac{N}{N+1}\right),$$

and the noise coefficients ${\xi }_{j,k}$ have the standard normal distribution, but are not independent. The variances of $W_{j,k}$ have the form ${\sigma }_j^2=2^{(1-\alpha )(J-j)}$ [12]. To suppress the noise the coefficients $W_{j,k}$ are processed with the hard thresholding function ${\rho }_H(y,T)=x\mathbf{1}\left(\right|y|>T)$ or the soft thresholding function ${\rho }_S(y,T)=\mathrm{sgn}\left(x\right){\left(\left|y\right|-T\right)}_{+}$ with some threshold T, and the estimates ${\widehat{W}}_{j,k}$ are obtained. After that, the inverse wavelet transform is performed. The idea of threshold processing is that the wavelet transform provides a “sparse” representation of the useful signal function; i.e., the signal is represented by a relatively small number of modulo large coefficients. To provide a “sparse” representation of a function that is uniformly Lipschitz regular with an exponent $\gamma$, the wavelet function participating in the discrete wavelet transform must have M continuous derivatives ($M\ge \gamma$) and M vanishing moments. It also must decrease fast enough at infinity. It is further assumed that the Meyer wavelets [1] that satisfy all the necessary conditions are used to perform the wavelet transform.

If we apply the discrete wavelet transform to the sample (1), we obtain the set of empirical wavelet coefficients

$$V_{j,k}={\nu }_{j,k}+{\xi }_{j,k},j=0,\dots ,J-1,k=0,\dots ,2^j-1.$$

Here ${\nu }_{j,k}$ are the coefficients of the discrete wavelet transform of the sample

$$f\left(x_{\left(1\right)}\right),\dots ,f\left(x_{\left(N\right)}\right).$$

In general, $V_{j,k}$ are not equal to $W_{j,k}$, and ${\nu }_{j,k}$ are not equal to ${\mu }_{j,k}$. However, one can apply the same thresholding procedure to the coefficients $V_{j,k}$ as to the coefficients $W_{j,k}$ and obtain the estimators ${\widehat{V}}_{j,k}$. The following sections discuss the properties of the resulting estimators.

3. Mean-Square Thresholding Risk

The mean-square thresholding risk for a sample with random grid is defined as

$$R_{\nu }(f,T)=\sum _{j=0}^{J-1}\sum _{k=0}^{2^j-1}\mathsf{E}{({\widehat{V}}_{j,k}-{\mu }_{j,k})}^2.$$

(3)

We also define the mean-square risk for the equispaced sample as

$$R_{\mu }(f,T)=\sum _{j=0}^{J-1}\sum _{k=0}^{2^j-1}\mathsf{E}{({\widehat{W}}_{j,k}-{\mu }_{j,k})}^2.$$

The threshold selection is one of the main problems in threshold processing. For the class $\mathrm{Lip}(\gamma ,L)$, the threshold $T_{\gamma }={\sigma }_j\sqrt{\frac{4\alpha \gamma }{2\gamma +\alpha }ln{2}^j}$ (calculated for each j) is close to optimal [16]. Using the results of [7] (Theorem 3), we can estimate the rate of $R_{\mu }(f,T_{\gamma })$.

Theorem 1.

Let $\alpha >1/2$ and $f\in \mathrm{Lip}(\gamma ,L)$ on the segment $[0,1]$ with $\gamma >{(4\alpha -2)}^{-1}$. Then for the threshold $T_{\gamma }$ we have

$$R_{\mu }(f,T_{\gamma })\le C{2}^{\frac{2\gamma +\alpha -2\alpha \gamma }{2\gamma +\alpha }J}{J}^{\frac{2\gamma +2\alpha }{2\gamma +\alpha }},$$

where C is a positive constant.

Additionally, repeating the arguments of Theorem 1 in [15], it can be shown that similar statement is valid for $R_{\nu }(f,T_{\gamma })$ when $\gamma >max\{{(4\alpha -2)}^{-1},1/2\}$. Thus, the replacement of equally-spaced samples by random ones does not affect the upper estimate for the rate of the mean-square risk.

4. Properties of the Mean-Square Risk Estimate

Since expression (3) explicitly depends on the unknown values of ${\mu }_{j,k}$, it cannot be calculated in practice. However, it is possible to construct its estimate based only observable data. This estimate is determined by the expression

$${\widehat{R}}_{\nu }(f,T)=\sum _{j=0}^{J-1}\sum _{k=0}^{2^j-1}F[V_{j,k},T],$$

(4)

where $F[V_{j,k},T]=(V_{j,k}^2-{\sigma }^2)\mathbf{1}\left(\right|V_{j,k}|\le T)+{\sigma }^2\mathbf{1}\left(\right|V_{j,k}|>T)$ for the hard threshold processing and $F[V_{j,k},T]=(V_{j,k}^2-{\sigma }^2)\mathbf{1}\left(\right|V_{j,k}|\le T)+({\sigma }^2+T^2)\mathbf{1}\left(\right|V_{j,k}|>T)$ for the soft threshold processing [1,3].

Estimator (4) provides an opportunity to get an idea of the evaluation error for the function f, since it can be calculated using only the observable values $V_{j,k}$. The following statement establishes its asymptotic normality, that, in particular, allows constructing asymptotic confidence intervals for the mean-square risk (3).

Theorem 2.

Let $f\in \mathrm{Lip}(\gamma ,L)$ on the segment $[0,1]$ with $\gamma >max\{{(4\alpha -2)}^{-1},1/2\}$, $\alpha >1/2$, and let the Meyer wavelet satisfy the conditions listed above. Then for the hard and soft threshold processing we have

$$\mathsf{P}\left(\frac{{\widehat{R}}_{\nu }(f,T_{\gamma })-R_{\nu }(f,T_{\gamma })}{D_J}<x\right)\to \mathsf{\Phi }\left(x\right)whenJ\to \infty ,$$

where $\mathsf{\Phi }\left(x\right)$ is the distribution function of the standard normal law, $D_J^2=C_{\alpha }{2}^J$, and the constant $C_{\alpha }$ depends only on α and the wavelet type.

Remark 1.

In practice, one needs to know the constant $C_{\alpha }$. Unlike the case of independent observations, this constant depends on the chosen wavelet. The method of calculation of $C_{\alpha }$ is discussed in [12].

Proof. 

Let us prove the theorem for the hard threshold processing method. In the case of soft threshold processing, the proof is similar.

Along with ${\widehat{R}}_{\nu }(f,T_{\gamma })$, consider

$${\widehat{R}}_{\mu }(f,T_{\gamma })=\sum _{j=0}^{J-1}\sum _{k=0}^{2^j-1}F[W_{j,k},T_{\gamma }]$$

and write the difference ${\widehat{R}}_{\nu }(f,T_{\gamma })-R_{\nu }(f,T_{\gamma })$ in the form

$${\widehat{R}}_{\nu }(f,T_{\gamma })-R_{\nu }(f,T_{\gamma })={\widehat{R}}_{\mu }(f,T_{\gamma })-R_{\mu }(f,T_{\gamma })+\tilde{R},$$

where

$$\tilde{R}={\widehat{R}}_{\nu }(f,T_{\gamma })-{\widehat{R}}_{\mu }(f,T_{\gamma })-(R_{\nu }(f,T_{\gamma })-R_{\mu }(f,T_{\gamma })).$$

In [12] with the use of the results of [17,18,19] it is shown that

$$\mathsf{P}\left(\frac{{\widehat{R}}_{\mu }(f,T_{\gamma })-R_{\mu }(f,T_{\gamma })}{D_J}<x\right)\to \mathsf{\Phi }\left(x\right)\mathrm{when}J\to \infty .$$

Therefore, to prove the theorem, it suffices to show that

$$\frac{\tilde{R}}{2^{J/2}}\stackrel{\mathsf{P}}{\to }0\mathrm{when}J\to \infty .$$

Under the conditions $\gamma >max\{{(4\alpha -2)}^{-1},1/2\}$ and $\alpha >1/2$, by virtue of Theorem 1 and a similar statement for $R_{\nu }(f,T)$, we obtain that

$$\frac{R_{\nu }(f,T_{\gamma })-R_{\mu }(f,T_{\gamma })}{2^{J/2}}\to 0\mathrm{when}J\to \infty .$$

Set

$$j_0\approx \frac{\alpha }{2\gamma +\alpha }J+\frac{1}{2\gamma +\alpha }{log}_{2}J.$$

Let us represent ${\widehat{R}}_{\nu }(f,T_{\gamma })-{\widehat{R}}_{\mu }(f,T_{\gamma })$ as

$${\widehat{R}}_{\nu }(f,T_{\gamma })-{\widehat{R}}_{\mu }(f,T_{\gamma })=S_1+S_2,$$

where

$$S_1=\sum _{j=0}^{j_0-1}\sum _{k=0}^{2^j-1}\left(F[V_{j,k},T_{\gamma }]-F[W_{j,k},T_{\gamma }]\right),$$

$$S_2=\sum _{j=j_0}^{J-1}\sum _{k=0}^{2^j-1}\left(F[V_{j,k},T_{\gamma }]-F[W_{j,k},T_{\gamma }]\right).$$

Since for some constant $\tilde{C}>0$ we have

$$|F[V_{j,k},T_{\gamma }]|\le \tilde{C}{T}_{\gamma }^2,\left|F[W_{j,k},T_{\gamma }]\right|\le \tilde{C}{T}_{\gamma }^2\mathrm{a}.\mathrm{s}.,$$

(5)

then

$$\frac{S_1}{2^{J/2}}\stackrel{\mathsf{P}}{\to }0\mathrm{when}J\to \infty .$$

Next

$$S_2=\sum _{j=j_0}^{J-1}\sum _{k=0}^{2^j-1}\left(F[V_{j,k},T_{\gamma }]-F[W_{j,k},T_{\gamma }]\right)=\sum _{j=j_0}^{J-1}\sum _{k=0}^{2^j-1}(V_{j,k}^2-W_{j,k}^2)+$$

$$+\sum _{j=j_0}^{J-1}\sum _{k=0}^{2^j-1}(W_{j,k}^2-2{\sigma }^2)\mathbf{1}\left(\right|V_{j,k}|\le T_{\gamma },|W_{j,k}|>T_{\gamma })+$$

$$+\sum _{j=j_0}^{J-1}\sum _{k=0}^{2^j-1}(2{\sigma }^2-V_{j,k}^2)\mathbf{1}\left(\right|V_{j,k}|>T_{\gamma },|W_{j,k}|\le T_{\gamma })+$$

$$+\sum _{j=j_0}^{J-1}\sum _{k=0}^{2^j-1}(W_{j,k}^2-V_{j,k}^2)\mathbf{1}\left(\right|V_{j,k}|>T_{\gamma },|W_{j,k}|>T_{\gamma }).$$

(6)

Consider the sum ${\displaystyle \sum _{j=j_0}^{J-1}}{\displaystyle \sum _{k=0}^{2^j-1}}(V_{j,k}^2-W_{j,k}^2)$:

$$\sum _{j=j_0}^{J-1}\sum _{k=0}^{2^j-1}(V_{j,k}^2-W_{j,k}^2)=\sum _{j=j_0}^{J-1}\sum _{k=0}^{2^j-1}({\nu }_{j,k}^2-{\mu }_{j,k}^2)+2\sum _{j=j_0}^{J-1}\sum _{k=0}^{2^j-1}{\xi }_{j,k}({\nu }_{j,k}-{\mu }_{j,k}).$$

Using the results of [12,15,20], it can be shown that the conditional distribution of this sum for fixed $x_i$ is normal with the mean

$$\sum _{j=j_0}^{J-1}\sum _{k=0}^{2^j-1}({\nu }_{j,k}^2-{\mu }_{j,k}^2)$$

and the variance that is less than

$${\tilde{C}}_{\alpha }\sum _{j=j_0}^{J-1}\sum _{k=0}^{2^j-1}{({\nu }_{j,k}-{\mu }_{j,k})}^2,$$

where ${\tilde{C}}_{\alpha }$ is a positive constant.

Since $f\in \mathrm{Lip}(\gamma ,L)$, repeating the arguments of [20], it can be shown that

$$\frac{1}{2^{J/2}}{\mathsf{E}}_x\left|\sum _{j=j_0}^{J-1}\sum _{k=0}^{2^j-1}({\nu }_{j,k}^2-{\mu }_{j,k}^2)\right|\to 0,$$

$$\frac{1}{2^{J/2}}{\mathsf{E}}_x\sum _{j=j_0}^{J-1}\sum _{k=0}^{2^j-1}{({\nu }_{j,k}-{\mu }_{j,k})}^2\to 0.$$

(7)

Hence, applying the Markov inequality, we obtain

$$\frac{1}{2^{J/2}}\sum _{j=j_0}^{J-1}\sum _{k=0}^{2^j-1}({\nu }_{j,k}^2-{\mu }_{j,k}^2)\stackrel{\mathsf{P}}{\to }0,$$

$$\frac{1}{2^{J/2}}\sum _{j=j_0}^{J-1}\sum _{k=0}^{2^j-1}{({\nu }_{j,k}-{\mu }_{j,k})}^2\stackrel{\mathsf{P}}{\to }0$$

when $J\to \infty$. Thus,

$$\frac{{\displaystyle \sum _{j=j_0}^{J-1}}{\displaystyle \sum _{k=0}^{2^j-1}}(V_{j,k}^2-W_{j,k}^2)}{2^{J/2}}\stackrel{\mathsf{P}}{\to }0\mathrm{when}J\to \infty .$$

The remaining sums in (6) contain indicators where either $|V_{j,k}|>T_{\gamma }$ or $|W_{j,k}|>T_{\gamma }$. Repeating the reasoning from [12] and using (7), it can be shown that, when divided by $2^{J/2}$, they also converge to zero in probability. The theorem is proven. □

Theorem 2 provides the possibility to construct asymptotic confidence intervals for the mean-square thresholding risk on the basis of its estimate.

In addition to the asymptotic normality, the estimator (4) also possesses the property of strong consistency.

Theorem 3.

Suppose that the conditions of Theorem 2 are satisfied. Then for hard or soft threshold processing, for any $\lambda >1/2$ we have

$$\frac{{\widehat{R}}_{\nu }(f,T_{\gamma })-R_{\nu }(f,T_{\gamma })}{2^{\lambda J}}\to 0a.s.whenJ\to \infty .$$

Since (5) holds, for fixed $x_i$ the conditional version of Bosq inequality [21] (Theorem 1.3) applies for (4), and the proof of this statement almost completely repeats the proof of the corresponding risk estimator property in [13].

5. Discussion

As it has been already mentioned, Theorem 2 provides the possibility to construct asymptotic confidence intervals for the mean-square thresholding risk. For practical purposes, it is desirable to have guaranteed confidence intervals. These intervals could be constructed based on the estimates of the convergence rate in Theorem 2. The estimates should depend on the Lipschitz parameters and parameter $\alpha$. Guaranteed confidence intervals would help to explain how the results of Theorems 2 and 3 affect the error estimation for the finite signal size. We therefore leave the problem of estimation of the rate of convergence and explicit numerical simulation for future work.

The obtained results are applicable to Meyer wavelets. Their advantage is that they possess infinitely many vanishing moments. It simplifies the proof of asymptotic normality in [12]. In view of the results of [8] it is clear that similar conclusions could be obtained with other wavelets that have a large enough number of vanishing moments (e.g., various Daubechies families).

It follows from Theorems 2 and 3 that the statistical properties of the mean-square risk estimator in a model with the uniform random design remain the same as in a model with equispaced samples. Note that this situation is not common. Random times of sample registration can also result in a random sample size. This situation was considered in [22]. In this case, the properties of the model can significantly differ from the properties of the fixed sample size model. For example, the limit distribution of the mean-square risk estimator can be a scale mixture of normal laws, which can have significantly heavier tails than the normal distribution. In particular, this distribution may belong to the class of stable laws, and it is well known that the variances of all stable laws, except the normal one, are infinite (the properties of stable distributions are discussed in detail in the monograph of V. M. Zolotarev [23]; see also [24]).