An FCM-Based Image De-Noising with Spatial Statistics Pilot Study

Abstract

Image de-noising is an important scheme that makes an image visually prominent and obtains enough useful information to produce a clear image. Many applications have been developed for effective noise suppression that produce good image quality. This study assumed that a residual image consisted of noise with edges produced by subtracting the original image with a low-pass-filter-smoothed image. The Moran statistics were then used to measure the variation in spatial information in residual images and we then used this information as feature data input into the Fuzzy C-means (FCM) algorithm. Three clusters were pre-assumed for FCM in this work: they were heavy, medium, and less noisy areas. The rates for each position partially belonged to each cluster determined using an FCM membership function. Each pixel in a noisy image was assumed in de-noising processing as a linear combination of the product of three de-noised images with membership functions in the same position. Average filters with different windows and a Gaussian filter were a priori applied to this noisy image to create three de-noised versions. The results showed that this scheme worked better than the non-adaptive smoothing. This scheme‘s performance was evaluated and compared to the bilateral filter and non-local means (NLM) using the peak signal to noise ratio (PSNR) and structure similarity index measure (SSIM). The developed scheme is a pilot study. Further future studies are needed on the optimized number of clusters and smoother versions used in linear combination.

1. Introduction

Noise is a major source of digital image contamination. Image noise may arise from image data quantization, transmission errors, electronic interference from the imaging hardware, as well as other sources [1]. The noise in digital images affects the real image signal, resulting in an image quality decline. This influence will affect the performance of digital image application areas such as segmentation, retrieval, edge extraction, etc. To obtain reliable results, efforts have been made in many applications to achieve effective noise suppression. Image de-noising, in the field of image processing, is a commonly used method to make the image visually prominent and obtain enough useful information [2,3].

The major function of image de-noising is to recover the original image from a noisy measurement and retain the image structure information as much as possible [4]. De-noising is a well-known method used from time to time but produces an ill-posed problem in image processing. To date, many de-noising digital image schemes have been continuously developed, from pixel level filtering methods, such as Gaussian filtering, bilateral filtering, and total variation regularization, to patch level filtering methods, such as non-local means [3,5,6], block-matching 3D filtering (BM3D) [4], and low-rank regularization, etc. [3,7].

Tomasi et al. introduced bilateral filtering as a nonlinear filter that combines domain and range filtering [8]. This filter replaces each pixel with the weighted average of its neighbors. The weight assigned to each neighbor decreases with both the distance in the image plane and the distance on the intensity axis [6]. This filter produces a smooth image while preserving the edges. It demonstrated great effectiveness for a variety of computer vision and computer graphics problems [6]. The non-local means filtering proposed by Buades et al. [4] is a spatial image de-noising algorithm based on bilateral filtering. This de-noising method replaces each pixel in the noisy image using the weighted pixel average with the related surrounding neighborhoods. The weighting function is determined by the similarity between neighborhoods [9]. It was applied to image de-noising in various specific fields because of its superior performance over other methods, such as the NLM filter and bilateral filtering [9,10].

To estimate white Gaussian noise in images, a work surveyed six methods and found that the noise estimation using standard deviation measurement in residual images was most dependable [11]. The residual image was obtained by subtracting the original and a low-pass-filter-smoothed image. Chuang et al. measured the noise level directly from the residual image of a bit plane. They claimed that the noise level in the original image might be overestimated due to the nature of the quantization involved [12]. A residual image (RE) can be defined as RE = OR − SM using the subtraction value between an original (OR) image and its smoothed (SM) version. The subtraction can effectively remove the signal part and leave the noise with edge parts [12,13].

Many RE-based state-of-the-art de-noise algorithms have been proposed in recent decades. Baloch et al. developed an RE correlation regularization de-noising scheme that minimizes the correlation between neighboring RE patches [14]. They claimed that correlation-based regularization can help to produce better results than K-SVD (a dictionary learning algorithm), both qualitatively and quantitatively. However, a computational burden drawback exists in the algorithm. Wang et al. proposed a residual-based method that combined the bilateral filter and structure adaptive kernel filter for Gaussian noise de-noising [15]. The noise was suppressed efficiently and this combined method showed acceptable de-noising performance for heavy Gaussian noise. However, the iterations are a time-consuming task [1]. This method uses a novel criterion to determine wavelet coefficient thresholds by minimizing the computed RE image kurtosis. Roychowdhury et al. estimated noise in chest CT image data with varying image quality using RE [16].

The RE should possess the statistical properties of contaminating noise. However, it is very likely that the residual patch will contain remnants of the clean image patch [14]. As a result, at high noise levels, the RE usually contains structures from the clean image patch; thus, it does not contain contaminating noise. Brunet et al. applied a statistical test on the RE and found that the RE did not contain only pure noise, as there were structures present [13]. They de-noised the RE with an adaptive Wiener filter first and then parts of the cleaned RE, where one of the hypothesis tests was rejected, were added back into the de-noised image. The idea is to first smooth the image’s flat regions, and then work on the details. Their iterative scheme produced gains in both PSNR and SSIM.

Adaptive de-noising involves newly proposed algorithms that perform local image computation in contrast to global de-noising [17]. These schemes improve the image quality and edge conservation more effectively than the global methods [18,19,20,21,22,23,24]. The adaptive image de-noising filter has become a promising research area [20].

A nonlinear module, named the Moran statistic, showed a correlation with the noise levels of medical images [12]. The Moran’s Z measurements presented high consistency with the variation in smoothing and sharpening in images [25]. This calculation proved that the spatial correlation corresponds well with the variation in image spatial properties [26,27]. Both Chen et al. and Hung et al. recently proposed using the Moran statistic calculation as an index for image de-noising [25,26,27,28]. In their work, the Moran-statistic-based adaptive image filter algorithms produced better image quality than global methods.

Fuzzy C-means (FCM) clustering is an unsupervised scheme successfully applied to feature analysis, clustering, and classifier designs [29,30]. The FCM algorithm classifies the image by grouping similar data points in the feature space into clusters if the images are presented in “feature spaces”. The FCM algorithm assigns pixels to each category by using fuzzy memberships. Each point in the feature space partially belongs to the clusters using the fuzzy membership function [31]. FCM has been used to decrease contamination from a mixed combination of impulse and Gaussian noise on digital images. The results demonstrate that this noise reduction technique outperforms state-of-the-art filters with respect to the metric PSNR [32].

This work proposes estimating the smoothness and sharpness spatial information in RE images using the Moran statistic. The Moran’s Z measurements are then used as feature data input into FCM. Three clusters are pre-assumed for the FCM in this work: they are heavy noise, medium noise, and less noisy areas (i.e., structure areas). The rates for each position partially belong to the clusters determined by an FCM membership function. The membership function uses weights to calculate the weighted sum of each position. Each pixel in a noisy image is a linear combination of the product of three SMs with membership functions in the same position. The average filters with different windows (3 × 3, 5 × 5) and a Gaussian filter are a priori applied to this noisy image to obtain three SMs. The results show that this scheme works better than the non-adaptive schemes.

This scheme’s performance is evaluated in terms of the peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and comparison to the bilateral filter and NLM.

2. Methods and Materials

2.1. RE

The smoothed image is obtained by averaging the neighboring pixels in a mask, i.e., using a 3 × 3 or 5 × 5 average filter. The RE₃ can be defined as RE₃ = OR SM₃ using the subtraction value between an original (OR) image and its smoothed version (SM₃). The SM₃ is produced using a 3 × 3 window and SM₅ using a 5 × 5 window.

2.2. Moran Test

Spatial autocorrelation is defined as follows: “Spatial autocorrelation refers to the fact that the value of a variable at one point in space is related to the value of that same variable in a nearby location” [28,33]. The spatial information concept has long been used in image de-noising and image quality estimation with different format types [25,26,27,28]. To determine the spatial autocorrelation, Moran introduced AC to measure the degree of spatial autocorrelation in a data area [33,34]. AC is shown in the following equation, and its range is between −1 and 1:

$$AC=\frac{N{\displaystyle \sum }_{j=1}^{r\times c}{\displaystyle \sum }_{i=1}^{r\times c}{\delta }_{ij}(x_i-\stackrel{\_}{x})(x_j-\stackrel{\_}{x})}{S_0\sum _{i=1}^{r\times c}{(x_i-\stackrel{\_}{x})}^2}$$

(1)

where x_i is the gray level for pixel i, $\stackrel{\_}{x}$ is the mean window gray level, S₀ = 2(2 mn – m – n), m and n are the number of rows and columns in the window, N is the total number of pixels in the window, and δ_ij = 1 if pixel i and j are adjacent and 0 otherwise. A larger AC value means a greater correlation between the pixels and that the area is not likely noise. When the size of N is large enough (>25), the variable AC follows a normal distribution with the mean and variance given by

$$a=-1/(N-1)$$

(2)

and

$${\sigma }^2=\frac{N\left[\right(N^2-3N+3)S_1-N{S}_2+3S_0^2]-K\left[N\right(N-1)S_1-2N{S}_2+6S_0^2]}{(N-1)(N-2)(N-3)S_0^2}-a^2$$

(3)

where $K=N\sum (x_i-\stackrel{\_}{x}{)}^4/[\sum (x_i-\stackrel{\_}{x}{)}^2{]}^2$, S₁ = 2S₀, and S₂ = 8(8mn – 7m – 7n + 4). We can use the standardized normal statistic

$$Z=\frac{AC-a}{\sigma }$$

(4)

to determine the features of an image.

A higher Z will lead to null hypothesis rejection, making the image noisy. This means that more structured information exists in this area and random noise is less likely in the image [12]. Based on this premise, the Moran Z value has successfully been applied to present the image spatial features [18,19,25,26,27,35]. The Moran statistics used in this work divide the noisy areas in the RE.

2.3. Fuzzy C-Means (FCM)

FCM is a data clustering method. This technique groups a data set into N clusters with every data point. The fuzzy methods determine which data belong to which clusters in a certain degree; they often provide better results than definite methods [30]. FCM clustering is an unsupervised technique successfully applied to feature analysis, clustering, and image segmentation, and it is the most popular fuzzy method [31]. The FCM algorithm classifies the image by grouping similar data points in the feature space into clusters if the image is represented in various feature spaces. The FCM algorithm assigns pixels to each category using fuzzy memberships. Every pixel is assigned to each category partially using fuzzy memberships. Let X = (x₁, x₂,…, x_N) denote a feature space with N points to be partitioned into M clusters, where x_j represents a multispectral (feature) data point. The algorithm is an iterative optimization that minimizes the cost function defined as follows:

$$J={\sum }_{j=1}^N{\sum }_{i=1}^M{u}_{ij}^m\parallel x_j-c_i{\parallel }^2$$

(5)

where u_ij($\in [0, 1]$) indicates the degree that the sample x_j belongs to the cluster center c_i and ${\sum }_{i=1}^M{u}_{ij}=1$; the ‖·‖ is a norm metric. The fuzziness factor m is used to adjust the weighting effect of membership values.

The fuzziness of the resulting partition increases with m. In this study, m = 2 was used. c_i is the ith cluster center and it can be calculated as

$$c_i=\frac{{\sum }_{j=1}^N{u}_{ij}^m{x}_j}{{\sum }_{j=1}^N{u}_{ij}^m}$$

(6)

$$u_{ij}=\frac{1}{{\sum }_{k=1}^M{\left(\frac{\parallel x_j-c_i\parallel }{\parallel x_j-c_k\parallel }\right)}^{\frac{2}{m-1}}} m\ne 1$$

(7)

Starting with an initial guess for each cluster center, the FCM will converge to a solution for c_i representing the local cost function minimum. The iteration is stopped when the maximum difference between two cluster centers at two successive iterations is less than a threshold (=0.00001). FCM was implemented using Matlab, version 2017b (9.3.0.713579).

3. Experiments and Results

The RE should possess mainly noise. Nevertheless, it is likely that the RE contains remnants of the clean image [14]. The RE consists of both noise and edges. This means that each pixel in the RE contains both noise and edges. The RE shows more edges than noise if decreasing the passing degree when a low-pass filter is applied to the image and vice versa. A more blurred SM image can be obtained when we increase the window size of an average filter or decrease the passing rate. In the same way, this processing produces more noise and edges in an RE too.

3.1. Experiments

A noisy Barbara image, as shown in Figure 1, was used to demonstrate the above assumption. Gaussian white noise was padded to this image first (μ = 100, σ = 10, on 20% of the entire image padded randomly) and it was then filtered with a 3 × 3 and a 5 × 5 average filter to generate RE₃ and RE₅, respectively. Following this, we measured Moran’s Z in both REs.

Moran’s Z showed a spatial correlation, which corresponded well with the variation in image spatial properties. A higher Z value means less noise and more structures in an area and vice versa. The spatial properties shown using Moran’s Z histogram for RE₃ and RE₅ are different, as shown in Figure 2. Moran’s Z was measured using a 5 × 5 sliding window. It is obvious that the curve of RE₅ shifts to a higher Z area than RE₃. The result shows that there are more structure areas in RE₅ than in RE₃. The spatial autocorrelation scheme clarified the spatial information in the images.

These Moran’s Z measurements were then used as feature data in the FCM algorithm. Three clusters were pre-assumed for FCM in this work: they were heavy noise (lowest Z area), medium noise, and less noisy areas (highest Z area), as shown in Figure 3. This figure also shows three cluster center points in the feature domain, where the Moran’s Z values for RE₃ and RE₅ are coordinates. Each point in the feature data space has three membership rates. A brief flow chart for the proposed image de-noise scheme is shown in Figure 4.

3.2. Results

The proposed scheme assumes that the filtering process is a linear combination of the product of three SMs with membership functions. The average filters with two different windows (3 × 3, 5 × 5) and a Gaussian filter (σ = 0.5) were a priori applied to this noisy image to make three SMs. In this work, SM₅ was used for heavy noise, SM₃ for medium, and SM_0.5 for less noisy or edge areas.

Six frequently used images were selected in this work, as shown in Figure 5 (images were download from https://ccia.ugr.es/cvg/dbimagenes, accessed on 1 September 2022, Computer Vision Group, University of Granada). These test images were all sized 512 × 512 and 9 bits deep. Various Gaussian noise was a priori added to these images and they were then divided into three groups: (1) padded randomly on 20%, 50%, 75% of the entire image with μ = 0 and σ = 20, 30, 50, 60; (2) padded randomly on 50% of the entire image with μ = 30 and σ = 20, 30, 50, 60; (3) padded randomly on 20% of the entire image with μ = 100 and σ = 10, 15, 20, 30, respectively.

The average filter with window (3 × 3, 5 × 5), a Gaussian filter (σ = 0.5), the bilateral filter [36], the non-local means scheme [37], and the proposed algorithm were applied to these noisy images. The PSNR and SSIM [38] were calculated and are shown in Figure 6, Figure 7, Figure 8, Figure 9, Figure 10 and Figure 11.

Figure 6, Figure 7 and Figure 8 show the PSNR results. Figure 9, Figure 10 and Figure 11 show the SSIM results on the de-noised images. In these figures, (a, f): added 20%, μ = 0 with various σ, (b, g): added 50%, μ = 0 with various σ, (c, h): added 75%, μ = 0 with various σ, (d, i): added 50%, μ = 30 with various σ, (e, j): added 20%, μ = 100 with various σ.

The image qualities depend on the PSNR and SSIM values. For both indices, the higher values correspond to better quality. The higher σ indicates noise variations in the image. This variation can reduce the image quality. The abscissas in Figure 6, Figure 7, Figure 8, Figure 9, Figure 10 and Figure 11 show variations in σ. Both the PSNR and SSIM indices show declines in response to the increasing variation in σ.

In some of the proposed scheme’s results, a linear combination of SM₅, SM₃, and SM_0.5 is superior to the single de-noise filter. We can see this in Figure 8, Figure 9, Figure 10 and Figure 11. Most of the proposed scheme’s results are better than the bilateral filter and some are even better than NLM. The quality indices resulting from this novel scheme are not always superior to the single filter but most are satisfactory.

4. Discussion

4.1. Noise Levels with Membership Function

The scheme in this work assumes that the filtering process is a linear combination of the products of three SMs with membership functions. The lower Z areas are assigned to heavy noise, middle areas are assigned to medium noise, and higher Z areas are assigned to the edges or less noisy areas. Each pixel in the noisy image is assumed to partially belong to three partitions and has membership partition rates in the same positions. SM₅ is used for heavy noise and creates a product with the left area rates. SM₃ and SM_0.5 are used for medium, less noisy areas, or edges and create a product with the rates from these areas. These areas are shown in Figure 3.

To prove that the above assumption is correct, if the SMs corresponding to the originally assigned noisy areas are changed (i.e., arbitrarily exchanging SM₅, SM₃, SM_0.5 to any area), a product is obtained with the rates from these areas. A noisy Barbara image (μ = 100, σ = 10, 20% randomly added) was filtered with a 3 × 3 and a 5 × 5 average filter and a Gaussian filter (σ = 0.5) was applied to this noisy image to make three SMs, respectively. The results from both PSNR and SSIM for all exchanged cases were obviously worse than those obtained before. This effect is consistent with the previous reports on Moran statistics [34].

However, if the SM₅ assigned to heavily noisy areas and SM₃, SM_0.5 are used to obtain a product with the rates from the left areas or middle areas or exchanged using the same SM in two areas, the PSNR and SSIM results for all cases are slightly lower than or equal to those obtained before. This indicates that proper SM_s selection is important.

4.2. Number of Clusters

Three clusters are pre-assumed for FCM in this plot study. There are heavy, medium, and less noisy or edge areas, as shown in Figure 3. A lower number of clusters should make the de-noised image approach results resemble those of a single filter. The higher the number of clusters, the better the results. However, ensuring proper de-noise scheme selection and applying it correctly is a difficult issue.

4.3. Comparison of Schemes by Metrics

To demonstrate that the de-noising effect for the proposed scheme is good, four tables were created for clear comparison.

Table 1. PSNR and SSIM for Barbara using various de-noising schemes. We used random padded Gaussian white noise μ = 100, σ = 15, on 20% of the entire image. Ave3 = 3 × 3, Ave5 = 5 × 5 average filter, G05 = Gaussian filter (σ = 0.5), Bi = bilateral filter, NLM = non-local means.

Scheme	Ave3	Ave5	Proposed	G05	Bi	NLM
PSNR	25.4	25.4	25.4	24.4	21.8	22.7
SSIM	0.29	0.24	0.33	0.33	0.28	0.33

shows the PSNR and SSIM for a Barbara image using various de-noise schemes. In this image, we padded Gaussian white noise randomly into the image. The proposed scheme shows superiority or is the same as other filters for PSNR or SSIM. This is equal for the Lena image shown in

Table 2. PSNR and SSIM for Lena using various de-noising schemes. We used random padded Gaussian white noise μ = 0, σ = 20, on 30% of the entire image. Ave3 = 3 × 3, Ave5 = 5 × 5 average filter, G05 = Gaussian filter (σ = 0.5), Bi = bilateral filter, NLM = non-local means.

Scheme	Ave3	Ave5	Proposed	G05	Bi	NLM
PSNR	37.3	35.1	38.6	34.1	32.5	32.8
SSIM	0.48	0.42	0.53	0.54	0.45	0.46

.

Table 3. Average PSNR and SSIM for Barbara using various de-noising schemes. We used random padded Gaussian white noise, μ = 0, 0, 0, 30, 100; σ = 10, 15, 20, 30, 50, 60, on 20%, 50%, 75% of the entire image. There were 20 images in total. Ave3 = 3 × 3, Ave5 = 5 × 5 average filter, G05 = Gaussian filter (σ = 0.5), Bi = bilateral filter, NLM = non-local means.

Scheme	Ave3	Ave5	Proposed	G05	Bi	NLM
PSNR	28.3	27.7	28.7	27.8	26.7	27.4
SSIM	0.34	0.27	0.4	0.4	0.36	0.43

shows the average PSNR and SSIM for Barbara using various de-noising schemes. There were 20 images in total. The image was randomly padded with various Gaussian white noise, μ = 0, 0, 0, 30, 100; σ = 10, 15, 20, 30, 50, 60, on 20%, 50%, 75% of the entire image. The proposed scheme shows superior results to the other filters both for PSNR or SSIM. This is equal for the Lena image shown in

Table 4. Average PSNR and SSIM for Lena using various de-noising schemes. We used random padded Gaussian white noise, μ = 0, 0, 0, 30, 100; σ = 10, 15, 20, 30, 50, 60; on 20%, 50%, 75% of the entire image. There were a total of 20 images. Ave3 = 3 × 3, Ave5 = 5 × 5 average filter, G05 = Gaussian filter (σ = 0.5), Bi = bilateral filter, NLM = non-local means.

Scheme	Ave3	Ave5	Proposed	G05	Bi	NLM
PSNR	31.1	31.1	31	28.1	26.9	28
SSIM	0.34	0.27	0.4	0.4	0.36	0.43

.

5. Conclusions

This work proposed the estimation of the variation in spatial information in RE images using Moran statistics. The Moran’s Z measurements were used as feature data in FCM. Three clusters were pre-assumed for FCM in this work: they were heavy, medium, and less noisy or edge areas. The rates for each position partially belonged to clusters determined using an FCM membership function. The membership function used weights to calculate the weighted sum of each position. Each pixel in the noisy image assumed that it was a linear combination of the product of three SMs with membership functions in the same positions.

The average filters with two different windows (3 × 3, 5 × 5) and a Gaussian filter (σ = 0.5) were a priori applied to this noisy image to make three SMs. Six frequently used images were chosen in this work. Various Gaussian noise was a priori added to these images for testing and they were then de-noised using the proposed scheme.

The performance of this scheme was evaluated in terms of the PSNR, SSIM, and comparisons made to the bilateral filter and NLM. Results from the proposed scheme were superior to those of the single de-noising filter and most results were better than those of the bilateral filter and some better than the NLM. The quality indices resulting from this novel scheme were not always superior to those of the single filter, but most were good.

This scheme was a pilot study on image de-noising. In the future, further research on the optimized number of clusters and better smoothing or structured versions used in linear combination is needed to optimize this scheme.