This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (
This paper proposes the optimization relaxation approach based on the analogue Hopfield Neural Network (HNN) for cluster refinement of preclassified Polarimetric Synthetic Aperture Radar (PolSAR) image data. We consider the initial classification provided by the maximumlikelihood classifier based on the complex Wishart distribution, which is then supplied to the HNN optimization approach. The goal is to improve the classification results obtained by the Wishart approach. The classification improvement is verified by computing a cluster separability coefficient and a measure of homogeneity within the clusters. During the HNN optimization process, for each iteration and for each pixel, two consistency coefficients are computed, taking into account two types of relations between the pixel under consideration and its corresponding neighbors. Based on these coefficients and on the information coming from the pixel itself, the pixel under study is reclassified. Different experiments are carried out to verify that the proposed approach outperforms other strategies, achieving the best results in terms of separability and a tradeoff with the homogeneity preserving relevant structures in the image. The performance is also measured in terms of computational central processing unit (CPU) times.
In recent years, the increasing number of Polarimetric Synthetic Aperture Radar (PolSAR) sensors has been demanding solutions for different applications based on the data they provide. One of these applications is data classification to identify the nature of the different structures in the imaged surfaces, based on the microwave backscattered signal. Terrain and landuse classifications are important applications of PolSAR data, where many supervised and unsupervised classification methods have been proposed [
An important objective of the use of PolSAR data for quantitative remote sensing applications is to extract physical information from the observed scene. Cloude and Pottier [
Lee
In this paper, we propose a new method based on a Hopfield Neural Network (HNN) optimization approach, initially proposed by Hopfield and Tank [
An important advantage of the HNN optimization mechanism is its ability to build networks of nodes where each node is characterized by its state. The initial states of these nodes are obtained from the unsupervised Wishart approach. These states are then iteratively updated during the HNN process, taking into account the previous states and two types of external influences exerted by other nodes in its neighborhood. The external influences are mapped as two consistencies under the form of consistency coefficients. Both are embedded into an energy term, which is minimized also considering the states of the node to be updated and the states of its neighbors. At each iteration, under the HNN approach, the pixels in the image under classification are considered for a new cluster assignment,
The HNN process is an optimization approach trying to achieve a minimum energy value. Nevertheless, a tradeoff must be established between maximum cluster separability and minimum energy value. The definition of the consistency coefficients involved under the HNN approach constitutes the main contribution of this paper, inspired by the consideration that pixels that are near to the scene display similar properties and probably belong to the same region [
The paper is organized as follows. In Section 2 the HNN scheme is proposed. We give details about the complex Wishart classifier and the theory about the cluster separability measures, because they are required by the HNN process. The performance of the method is illustrated in Section 3, where a comparative study against the original maximum likelihood Wishartbased approach is carried out. Finally, Section 4 provides a discussion of some related topics and the final conclusions.
Before the HNN optimization process is applied, the Wishartbased classification process described in [
The polarimetric scattering information may be represented for each image pixel by the Pauli scattering vector
From the coherency matrices, we apply the H/ᾱ decomposition process as a refined scheme to parameterize polarimetric scattering problems. The scattering entropy,
The next step is to classify the PolSAR data into nine classes in the H/ᾱ plane, although zone three never contains pixels. These classes include different types of scattering mechanisms present in the scene, such as vegetation (grass, bushes), water surface (ocean or lakes) or city block areas. Section 3.3 includes a description and discussion about the content of these classes.
Hence, the classification process results in eight valid zones or clusters, where each class is identified as
Compute the distance measure for each pixel
Assign the pixel to the class with the minimum distance,
Verify if the termination criterion is met, otherwise set
Based on the above measures, the goal of the proposed reclassification is to achieve a small dispersion into the clusters and large distances between pairs of clusters, which leads to a small
Here, we introduce the HNN architecture and some preliminary considerations, its dynamical behavior, the energy definition and the summary of the HNN process.
At this stage, once the Wishart classifier has performed an initial classification, all the pixels have been classified into eight clusters or zones. A label identifying the zone is assigned to each pixel. The objective of the HNN is to perform a reclassification to modify the individual pixel’s labels based on consistency criteria with respect to the labels assigned to the surrounding pixels. We consider this relabeling as a region homogenization process. When a pixel changes its label, the pixel is assigned to a different class. This implies that their coherency matrices change and, consequently, the cluster centers also change according to
How can we achieve that a pixel changes its current label so that it is classified as belonging to a different class?
How can we achieve that a pixel does not change its label when its neighbors have identical labels as the label of the pixel under analysis?
How can we achieve maximum cluster separability?
When can we consider that no more changes are required?
The first question may be answered by considering that a pixel is classified to belong to a cluster if its distance to the corresponding cluster center is the minimum of all distances to all the cluster centers. Assuming that the pixel
Previously, we have considered the influence exerted by the pixels in the neighborhood over the central pixel. Nevertheless, not only the influence of the neighbors must be taken into account, but also the own effect of the central pixel in order to avoid an excessive influence of the neighbors. We call this influence the selfinfluence. The HNN is an energy optimization approach, which can embed contextual and selfinfluence, with the advantage that it can avoid local minima.
According to
For each cluster
As one may observe from
The HNN paradigm has been widely used for solving optimization problems. This implies fixing two characteristics [
The HNN is a recurrent network containing feedback paths from the outputs of the nodes back into their inputs, so that the response of such a network is dynamic. This means that after applying a new input, the output is calculated and fed back to modify the input. The output is then recalculated, and the process is repeated iteratively. Successive iterations produce smaller and smaller output changes, until eventually the outputs become constant,
The connection weights between the nodes in the network
For analog Hopfield networks, the total input into a node is converted into an output value by a sigmoid monotonic activation function instead of a thresholding operation employed in the case of discrete Hopfield networks [
The way to avoid that a continuous network cannot find a solution due to the existence of local minimum, and therefore to make the network converge to a solution state, is to decrease
The model provided in
The quantity describing the state of each network
The total energy for the
According to the results reported in [
The continuous Hopfield model described by the system of nonlinear firstorder differential
The HNN approach tries to achieve the most stable configuration for the network based on energy minimization. Now, the problem is to define the coefficients involved in the energy function, so that the network stability coincides with the minimum energy value. Hence, we need to define the meaning of stability for a node. A node is stable if its state remains invariable with iterations.
The term
The separation coefficient at the iteration
This coefficient embeds the concept of cluster separability defined in
Once the regularization and separation coefficients have been specified, we search for an energy function such that the energy is low when both consistencies are high and
Both coefficients are computed based on the influences exerted by the nodes in the neighborhood
Assuming contextual consistencies,
By comparison of the Expressions
According to the discussion in Section 2.3.2, to ensure the convergence to a stable state [
The energy in
As mentioned in the introduction, a DSA approach has also been applied to improve the Wishart classification in PolSAR data, where an energy function is also minimized with the same network architecture and identical consistency coefficients as the ones defined in
After mapping the energy function onto the Hopfield neural network, the image classification process is achieved by letting the networks evolve until they reach stable states,
Because the proposed HNN approach tries to achieve the maximum cluster separability based on the minimization of an energy function, the optimal convergence criterion should be: “maximum cluster separability with minimum energy”. Nevertheless, we have verified during our experiments that both are not met simultaneously. Indeed, whereas the energy generally decreases continuously, although in a smooth way, with the number of iterations, the averaged cluster separability coefficient
In order to assess the validity and performance of our proposed classification approach, we use the welltested NASA/JPL AIRSAR Lband image of the San Francisco Bay (SFB). The dimensions of the data are 900 × 1,024 pixels. This image displays several areas, including urban areas where thin structures induce neighboring pixels to belong to different clusters. The HNN tries to smooth these types of areas. Nevertheless, this information is preserved in the original image, either after the Wishart approach or even before the Wishart approach is triggered.
Additionally, we have considered a second PolSAR dataset at Cband. In this case, data were acquired by the EMISAR system property of the Danish Center for Remote Sensing on the Baltic Sea Lakes. These data are 512 × 512 pixels in size and display large targets, where the HNN tries to induce smoothing.
In order to assess the validity and performance of the proposed approach, we have designed the following test strategy. Because our proposed HNN approach starts after the iterative Wishart classification process, the first task consists of the determination of the best number of iterations suitable for the Wishart process. This is carried out by executing this process from one to
Once the best number of iterations is obtained for Wishart, as it is explained in Section 2.1, we fix the maximum number of iterations,
We tested the HNN for the following three neighborhood regions:
According to the above strategy,
As one can see, the best cluster separability is achieved for two iterations, because at this number we obtain the minimum averaged separability coefficient. This is the number of iterations employed for the Wishart classifier,
As one may observe from the results in
As we can see from
We compared our proposed HNN approach and the DSA in [
In order to evaluate the degree of homogeneity induced by the Wishart, the HNN, the DSA, the MAJ and the ICM approaches, we apply the following criterion: given a neighborhood, here
Results in
We also applied our HNN strategy for the Cband PolSAR image on the Baltic Sea Lakes (BSL).
From the results above, we have verified that both the HNN and the DSA approaches achieve similar performances in terms of separability but with an important nuance. The HNN approach requires less iterations than the DSA approach to achieve the convergence. For the SFB dataset, the HNN requires a unique iteration and the DSA two iterations, and for BSL two and three iterations. From
Three main consequences can be derived from this evidence: (a) Both optimization strategies (HNN and DSA) are suitable for improving the Wishart classification results; (b) the dynamical behavior of HNN is more effective than DSA in capturing both the mutual influences exerted by the neighborhood and the selfinfluence; (c) HHN does not require the definition of two parameters to establish the tradeoff of both mutual and selfinfluences.
HNN also outperforms ICM and MAJ in terms of averaged separability. This means that the greater the separabilities the better the classification decisions.
From the point of view of homogeneity, ICM and MAJ achieve slightly greater degrees of homogeneity. As the number of iterations increases, the classification results also display a high degree of homogenization. Although the general goal of the methods studied is designed toward homogenization, if this is excessive, the result is an overhomogenization, where relevant structures in the images could vanish. To deal with this problem, a tradeoff must be achieved between preserving as many relevant structures as possible and the image homogenization. This is solved by the HNN and DSA optimization strategies, because they consider not only the neighbors but also explicitly the own node or pixel which is being updated, both under the corresponding energy term, which for HNN is defined in
Another important support coming from HNN and DSA is that both processes can be controlled by energy minimization. As displayed in
As reported in the work of Lee
In [
The low entropy vegetation consisting of grass and bushes belonging to the cluster
The three distinct surface scattering mechanisms of the ocean surface identified in [
Also, in accordance with [
Some structures inside other broader regions are correctly isolated. This occurs in the rectangular area corresponding to a park, where the internal structures with high entropy are clearly visible [
Additionally, the homogenization effect can be considered as a mechanism for speckle noise reduction during the classification phase, avoiding the early filtering for classification tasks.
From the qualitative point of view the qualitative analysis is more difficult in the image BSL, because the original image displays large and relatively high homogeneity areas, but still we can see how ICM and MAJ tend to eliminate some structures, because the specific selfinfluences coming from the pixels to which they belong are ignored during the updating process and all pixels in the neighborhood contribute equally.
Finally, from
This paper focuses on the performance of the optimization Hopfield Neural Network (HNN) strategy as a suitable method to improve the Polarimetric Synthetic Aperture Radar (PolSAR) data classification results obtained by the standard Wishart classifier. The proposed methodology has been tested considering two different datasets: An Lband PolSAR dataset over the San Francisco Bay (SFB) and a Cband PolSAR dataset over the region of the Baltic Sea Lakes (BSL). HNN has been favorably compared against existing strategies including the Deterministic Simulated Annealing (DSA), which is also based on optimization, ICM and Majority. This comparison is based on the computation of averaged class separability values. We also obtain results, which improve the ones obtained by the standard Wishart classifier. For SFB, these values are 65.5 for HNN, 67.8 for DSA, 71.5 for ICM, 93.1 for MAJ and 78.3 for Wishart. For BSL, the values are 22.4 for HNN, 22.4 for DSA, 24.2 for ICM, 25.9 for MAJ and 24.3 for Wishart. In both cases, lower values indicate better performances.
We achieve homogenization while preserving the most important structures inside broader areas. The proposed approach could be helpful to other procedures, like the one proposed in [
HNN has a greater ability than DSA to capture and embed the information coming from the neighboring pixels and also from the own pixel under classification. Unlike DSA, HNN does not require two additional parameters to combine both types of information: (a) A constant representing the tradeoff between regularization and separation coefficients like the ones defined in
Although the proposed HNN and DSA approaches were designed for homogenization purposes, these optimization approaches are able to enhance specific features, such as buildings or trees in forests, by redefining the regularization coefficient. Indeed, instead of applying reinforcement for similarities of the states between the central pixel and its neighbors, i.e. maximum consistency, we can apply maximum consistency for dissimilarities. This arises when the emphasis is put on differentiation rather than on similarity. Thus, HNN and also DSA are sufficiently flexible in this respect. Moreover, due to the flexibility of the HNN, we can define as many contextual coefficients (constraints) as required with the intention to capture different effects. These coefficients should be conveniently embedded under the energy function according to
The main drawback of the proposed approach is its relatively high computational cost, which is also inherent to the five approaches involved in our experiments. In SFB these times in minutes are: Wishart (6.01), HNN (2.02), DSA (4.62), ICM (2.28) and MAJ (2.35). In BSL they are: Wishart (2.20), HNN (0.76), DSA (0.77), ICM (0.75) and MAJ (0.81).
HNN is also able to assume any type of separability measures like the Fischer linear discriminant analysis [
The effectiveness of this classification approach has been illustrated by the welltested NASA/JPL AIRSAR Lband data of the San Francisco bay, where detailed and specific scattering mechanisms are preserved. The second test with the Cband dataset has shown the capability to deal with large distributed areas resulting in a preservation of the polarimetric scattering information and a noticeable reduction of the speckle noise component.
In this paper, we have considered an initial classification derived from a target decomposition theorem based on the eigenanalysis of the coherency matrix. With the aim to favor the physical interpretation of the physical information that may be extracted from PolSAR data, in the future, additional efforts should be extended to consider alternative target decomposition theorems for classification based on a direct physical interpretation of the coherency matrix [
This work has been partially funded by National I+D project TEC201128201C0201. The authors would like to thank the NASA/JPL and the Danish Center for Remote Sensing for providing the data employed in this study. Thanks are due to the anonymous referees for their very valuable comments and suggestions.
Flowchart of the overall procedure and architecture for the Hopfield Neural Network (HNN) paradigm.
Averaged separability values for the first two iterations of the Wishart classifier and the best performance achieved with HNN and DSA during the four iterations according to Table.
Energy variation against the number of iterations for Wishart and HNN.
(
Expanded area corresponding to mountains extracted from
Expanded area corresponding to a city extracted from
(
Averaged separability values for the Wishart classifier against the number of iterations.
# of iterations  1  2  3  4  5  6  7 
91.8  116.1  145.2  112.8  93.0  106.6 
Averaged separability values
# of iterations  

1  2  3  4  5  6  7  8  

HNN  69.9  71.2  74.3  74.3  74.6  74.8  74.9  
 
DSA  76.6  69.1  70.0  70.0  70.3  70.4  70.5  
 

HNN  135.8  183.7  189.2  192.1  192.9  193.7  194.6  196.2 
DSA  177.3  158.9  182.2  188.1  190.5  197.1  199.4  202.6  
 

HNN  301.2  444.1  445.3  449.1  449.9  452.2  455.3  458.1 
DSA  354.3  321.4  432.6  488.4  489.9  493.2  495.1  497.3 
Averaged Cluster Separability values (
78.3  67.7  67.8  71.5  93.1  
0.354  0.286  0.291  0.192  0.184  
Average CPU times (minutes/iteration)  3.01  2.02  2.31  2.28  2.35 
Averaged Cluster Separability values (
24.3  22.4  22.4  24.2  25.9  
0.121  0.088  0.089  0.081  0.083  
Average CPU times (minutes/iteration)  1.10  0.76  0.77  0.75  0.81 