Hypergraph Embedding for Spatial-Spectral Joint Feature Extraction in Hyperspectral Images

Abstract

The fusion of spatial and spectral information in hyperspectral images (HSIs) is useful for improving the classification accuracy. However, this approach usually results in features of higher dimension and the curse of the dimensionality problem may arise resulting from the small ratio between the number of training samples and the dimensionality of features. To ease this problem, we propose a novel algorithm for spatial-spectral feature extraction based on hypergraph embedding. Firstly, each HSI pixel is regarded as a vertex and the joint of extended morphological profiles (EMP) and spectral features is adopted as the feature associated with the vertex. A hypergraph is then constructed by the K-Nearest-Neighbor method, in which each pixel and its most K relevant pixels are linked as one hyperedge to represent the complex relationships between HSI pixels. Secondly, the hypergraph embedding model is designed to learn a low dimensional feature with the reservation of geometric structure of HSI. An adaptive hyperedge weight estimation scheme is also introduced to preserve the prominent hyperedges by the regularization constraint on the weight. Finally, the learned low-dimensional features are fed to the support vector machine (SVM) for classification. The experimental results on three benchmark hyperspectral databases are presented. They highlight the importance of spatial–spectral joint features embedding for the accurate classification of HSI data. The weight estimation is better for further improving the classification accuracy. These experimental results verify the proposed method.

1. Introduction

Hyperspectral imaging is an important mode of remote sensing imaging, which has been widely used in a diverse range of applications, including environment monitoring, urban planning, precision agriculture, geological exploration, etc. [1,2,3]. Most of these applications depend on the key problem of classifying the image pixels within hyperspectral imagery (HSI) into multiple categories, i.e., HSI classification, and extensive research efforts have been focused on this problem [4,5,6,7,8,9].

In HSI, each pixel contains hundreds of spectral bands from the visible to the infrared range of the electromagnetic spectrum. In general, the spectral signature of each pixel can be directly used as the feature for classification. However, due to the noise corruption and high correlation between spectral bands, the using of the spectral feature alone is often unable to obtain good classification results. It is well accepted that the HSI pixels within a small spatial neighborhood are often made up of the same materials. Thus, spatial contextual information is also useful for classification [10,11]. Landgrebe and Ketting proposed the well-known extraction and classification of homogeneous objects (ECHO) approach that partitioned the HSI pixels into homogeneous object and classified homogeneous object as different categories [12]. Later, Markov random field (MRF) modeling was widely adopted to capture the interpixel dependency through the neighbor system [13,14]. However, the optimization of MRF-based methods is very time-consuming. Due to the high dimensionality of HSI data, the computationally effective algorithm is desirable. In this sense, Pesaresi and Benediktsson [15] proposed the use of morphological transformations to build a morphological profile (MP) for extracting the structural information. Palmason et al. [16] extended the method proposed in [15] to the high-resolution hyperspectral data classification. They first extracted several principal components of the hyperspectral data. Then, the MP is constructed based on each selected principal component. At last, all MPs are jointed as extended MP (EMP), which is input into a neural network for classification. However, EMP was primarily designed for classification of urban structures and it did not fully utilize the spectral information in the data. Regrading this issue, Fauvel et al. [17] proposed fusing the morphological information and the original hyperspectral data, i.e., the two vectors of attributes are concatenated into one feature vector. The final classification is achieved by using a support vector machine classifier. Many other spectral and spatial joint features [18,19,20,21,22], such as 3D wavelet [18], spatial and spectral kernel [19], matrix-based discriminant subspace analysis [20], etc. are used for classification.

These joint features usually have a high dimension. In order to avoid the Hughes phenomenon, feature extraction and dimensionality reduction must be conducted before classification. Principal component analysis (PCA) and Fisher’s linear discriminant analysis (LDA) [23] are two simple and effective approaches for dimension reduction. PCA aims at projecting the data along the directions of maximal variance. LDA is designed to generate the optimal linear projection matrix by maximizing the between-class distance while minimizing the within-class distance. Apart from these linear methods, many nonlinear versions have been developed, such as kernel PCA [24] and kernel LDA [25]. Some other feature extraction techniques have also been proposed, e.g., locality preserving projection (LPP) [26], independent component analysis (ICA) [27,28], and locally linear embedding (LLE) [29]. In particular, Yan et al. [30] proposed a general graph embedding (GE) model that seamlessly includes many existing feature extraction techniques. In this GE model, each data point is visualized as a vertex and a pairwise edge is used to represent the association relationship between two data points. They consider each feature extraction algorithm as an undirected weighted graph that describes geometric structures of data. GE algorithms have been widely explored for dimension reduction of HSI. Besides the geometric structures of data, sparsity is also explored to construct the graph embedding model. Luo et al. proposed constructing a graph with the sparse coefficients that reveals the sparse properties of data, and the transformation matrix is obtained for feature reduction [31]. In addition, by regarding different band sets as different views of land covers, multiview graph ensemble-based graph embedding is also utilized to promote the performance of graph embedding for hyperspectral image classification [32].

A hypergraph is a generalization of a pairwise graph. Different from pairwise graphs, each edge in a hypergraph is capable of connecting more than two vertices [33]. Thus, the complex relationships of the dataset can be captured by a hypergraph, and hypergraphs have been gaining more and more attention in recent years. Bu et al. [34] presented a hypergraph learning based music recommendation method with the use of hyperedges to exploit the complex social media information. A hypergraph semi-supervised learning model [35] was also proposed for image classification. Yuan et al. [36] utilized a hypergraph embedding model for HSI feature reduction, in which the spatial hypergraph models (SHs) are construed by selecting the K-nearest neighbors within the spatial region of the centroid pixel. Experimental results demonstrated that SH outperformed many existing feature extract methods for HSI classification, including raw spectral feature (RAW), PCA, LPP, LDA, nonparametric weighted feature extraction (NWFE) [37] and semi-supervised local discriminant analysis (SELD) [38]. However, SH is designed to learn the projection matrix for reducing the spectral feature. The spatial structure is not exploited for hypergraph embedding, which is not capable of simultaneously extracting the spectral-spatial features. Furthermore, the hyperedge weight is computed in advance and fixed in the hypergraph embedding procedure. As the discussion stated in [39,40], all of the hyperedges do not have the same effect on the learning procedure. Some hyperedges are not as informative as others. The hypergraph embedding should be enhanced by estimating the hyperedge weights adaptively.

In order to cope with these issues, we propose a novel algorithm for HSI spatial-spectral joint feature extraction. We combine the EMP and spectral features and adopt the KNN method to construct a hypergraph, where each sample and its K nearest neighbors are enclosed in one hyperedge. Similar to [36], a linear projection matrix P can be learnt by solving the hypergraph embedding model. However, in [36], the hyperedges’ weights in the hypergraph embedded model are fixed. Inspired by [39,40], we introduce a scheme to update the weights adaptively to preserve the prominent hyperedge and further learn the low-dimensional structure. It helps improve the accuracy of the final HSI classification to a certain extent. Finally, the leaned low-dimensional features are fed to the SVM for classification. The flowchart of the proposed method is shown in Figure 1. Experiments conducted on three widely used types of HSI demonstrate that the proposed method achieves superior performance over many other feature extract methods for HSI classification.

2. Hypergraph Model

Denote a hypergraph as $\mathbf{G}=(\mathbf{V},\mathbf{E},\mathbf{W})$, which consists of a set of vertices V, a family of hyperedge E and a weight matrix W of hyperedges. Different from pairwise graphs (For convenience, we call it a simple graph in the following), every hyperedge $e_i$ can contain multiple vertices and is assigned a weight $w\left(e_i\right)$. As shown in Figure 2b, hyperedge $e_1$ is composed of vertices $v_1$, $v_2$ and $v_3$. $e_2$ is composed of vertices $v_3$ and $v_4$. $e_3$ is composed of vertices $v_4$, $v_5$, $v_6$ and $v_7$. W is a diagonal matrix of the hyperedge weights. The connection relationship of hypergraph G can be represented by an incidence matrix $\mathbf{H}\in {\mathbf{R}}^{\left|\mathbf{V}\right|\times \left|\mathbf{E}\right|}$, which can be defined as:

$$H_{ij}=\mathbf{H}(v_i,e_j)=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}{v}_i\in e_j,\hfill \\ 0,\hfill & \mathrm{if}{v}_i\notin e_j.\hfill \end{array}\right.$$

(1)

The degree of vertex v and hyperedge e can be respectively represented as:

$$d\left(v_i\right)=\sum _{e_j\in \mathbf{E}}w\left(e\right)\mathbf{H}(v_i,e_j),$$

(2)

$$\delta \left(e_j\right)={\delta }_j=\sum _{v_i\in \mathbf{V}}\mathbf{H}(v_i,e_j).$$

(3)

According to the above definition, the main difference between hypergraphs and simple graphs is that every hyperedge can link more than two vertexes. Therefore, hypergraph is suitable to represent local group information and the high-order relationship of data. For example, considering seven vertices in Figure 2b, they are attributed to three groups and the corresponding incidence matrix is shown in Figure 2c. In terms of building a simple graph with these seven data points, the complex relations within the group are broken into multiple pairwise links. Some valuable information may be lost in this procedure; therefore, a simple graph can not describe the group structure well.

3. Hypergraph Embedding of Spatial-Spectral Joint Features

As shown in Figure 1, our algorithm mainly consists of three steps: spatial-spectral joint feature construction, hypergraph embedding and SVM classification.

3.1. Spatial-Spectral Joint Feature Construction

Following [16], we first extract several PCs from the original HSI $\mathbf{I}\left(x\right)$ and then build an MP from each of the PCs:

$$\mathbf{MP}\left(x\right)=\left\{{\mathbf{CP}}_n\left(x\right),\dots ,\mathbf{I}\left(x\right),\dots ,{\mathbf{OP}}_n\left(x\right)\right\},$$

(4)

where n is the number of the circular structural element (SE) with different radius sizes, ${\mathbf{OP}}_n\left(x\right)$ and ${\mathbf{CP}}_n\left(x\right)$ are the opening profile (OP) and the closing profile (CP) at the pixel x with an SE of a size n, respectively. Specifically, we have ${\mathbf{CP}}_0\left(x\right)={\mathbf{OP}}_0\left(x\right)=\mathbf{I}\left(x\right)$. The MP of $\mathbf{I}$ contains the original image $\mathbf{I}$, n opening profile and n closing profile. Therefore, each MP is a $(2n+1)$-dimensional vector. Finally, all MPs are stacked together in one as EMP:

$$\mathbf{EMP}\left(x\right)=\left\{{\mathbf{MP}}_{{\mathrm{PC}}^1}\left(x\right),{\mathbf{MP}}_{{\mathrm{PC}}^2}\left(x\right),\dots ,{\mathbf{MP}}_{{\mathrm{PC}}^m}\left(x\right)\right\},$$

(5)

where m represents the number of PCs. The EMP is defined as an $m(2n+1)$-dimensional vector.

After obtaining the EMP feature, we represent the spatial and spectral joint feature of the i-th HSI pixel as

$$v_i=\left[\begin{array}{c}{x}_i\hfill \\ \mathrm{EMP}\left(x_i\right)\hfill \end{array}\right]\in R^{\mathrm{m}(2\mathrm{n}+1)+d},$$

(6)

where d is the number of the spectral bands. Denote the spectral features matrix of HSI as $\mathbf{X}=[{\mathbf{x}}_1,{\mathbf{x}}_2,\dots ,{\mathbf{x}}_N]\in {\mathbf{R}}^{d\times N}$, EMP matrix of HSI as $\mathbf{EMP}=\left[\mathrm{EMP}\left(x_1\right),\cdots ,\mathrm{EMP}\left(x_N\right)\right]$, where $x_i$ is the i-th pixel, and N is the number of HSI pixels. Then, the joint feature matrix of HSI can be represented as: $\mathbf{V}=\left[\begin{array}{c}\mathbf{X}\hfill \\ \mathbf{EMP}\hfill \end{array}\right]\in R^{\left(\mathrm{m}\right(2\mathrm{n}+1)+d)\times N}$.

3.2. Hypergraph Embedding

We take each pixel of HSI as a vertex and construct a hypergraph $\mathbf{G}=(\mathbf{V},\mathbf{E},\mathbf{W})$ to represent the correlation between HSI pixels. Each vertex $v_i$ is associated with the spatial and spectral joint feature defined in Equation (6). The hypergraph G is constructed by the K-nearest neighbor method. In detail, each pixel ${\mathit{v}}_i$ and its K nearest neighbors are enclosed as hyperedge $e_i$. Thus, hyperedge set $\mathbf{E}=\{e_1,e_2,\dots ,e_N\}$ contains N hyperedges. Meanwhile, the weight $w\left(e_i\right)$ of hyperedge $e_i$ is defined as:

$$w\left(e_i\right)=\sum _{v_i,v_j\in e_i}\exp \left(-\frac{{∥v_j-v_i∥}_2^2}{2{\sigma }^2}\right),$$

(7)

where $\sigma$ is the mean distance between all vertices and can be calculated by $\sigma =\frac{1}{N^2}{\displaystyle \sum _i}{\displaystyle \sum _j}d\left({\mathit{v}}_i,{\mathit{v}}_j\right)$, $d\left({\mathit{v}}_i,{\mathit{v}}_j\right)$ is the distance between vertex ${\mathit{v}}_i$ and vertex ${\mathit{v}}_j$. The degree of vertex ${\mathit{v}}_i$ and the degree of hyperedge $e_i$ can be computed by Equations (2) and (3), respectively. Based on this definition, the more "compact" hyperedge (local group) is assigned with a higher weight.

Denote ${\mathbf{D}}_v$ and ${\mathbf{D}}_e$ as two diagonal matrices of the vertex degrees and the hyperedge degrees, respectively, and $\mathbf{P}\in {\mathbf{R}}^{\left(m\left(2n+1\right)+d\right)\times u}$ (generally, $m\left(2n+1\right)+d>>u$) as the linear projection matrix. The objective of hypergraph embedding model is to learn the projection matrix $\mathbf{P}$ for reducing the feature dimension with the preservation of geometric property in the original space. The objective function is formulated as:

$$\begin{array}{c}\underset{{\mathbf{P}}^T\mathbf{V}{\mathbf{D}}_v{\mathbf{V}}^T\mathbf{P}=1}{\min }\frac{1}{2}{\displaystyle \sum _{e\in \mathbf{E}}}{\displaystyle \sum _{v_i,v_j\in e}}\frac{w\left(e\right)h\left(v_i,e\right)h\left(v_j,e\right)}{\delta \left(e\right)}{∥{\mathbf{P}}^T{v}_i-{\mathbf{P}}^T{v}_j∥}_2^2\hfill \\ =\frac{1}{2}{\displaystyle \sum _{k=1}^N}{\displaystyle \sum _{i,j=1}^N}\frac{w_k{h}_{ik}{h}_{jk}}{{\delta }_k}{∥{\mathbf{P}}^T{v}_i-{\mathbf{P}}^T{v}_j∥}_2^2\hfill \\ =\mathrm{trace}\left({\mathbf{P}}^T\mathbf{VL}{\mathbf{V}}^T\mathbf{P}\right),\hfill \end{array}$$

(8)

where $\mathbf{L}={\mathbf{D}}_v-{\mathbf{HWD}}_e^{-1}{\mathbf{H}}^T$ is the hypergraph laplacian matrix. The constraint ${\mathbf{P}}^T\mathbf{V}{\mathbf{D}}_v{\mathbf{V}}^T\mathbf{P}=1$ is used for scale normalization of the low-dimensional representations. This objective function induces the constraint that if $v_i$ and $v_j$ are similar and belong to the same hyperedge, they should also be adjacent in embedded space. In addition, an efficient hypergraph weight estimation scheme is proposed to preserve the prominent hyperedges. Assuming that $\mathit{w}={\left(w_1,w_2,\dots ,w_N\right)}^T$ is composed of the elements lying in the main diagonal of W, we enforce ${\mathbf{1}}_N^T\mathit{w}=1$ and add an $l_2$ norm regularizer on $\mathit{w}$. Then, our proposed embedding model is finally defined as:

$$\left\{{\mathbf{P}}^{*},w^{*}\right\}=\arg \underset{{\mathbf{P}}^T\mathbf{V}{\mathbf{D}}_v{\mathbf{V}}^T\mathbf{P}=1}{\min }\left\{\mathrm{trace}\left({\mathbf{P}}^T\mathbf{VL}{\mathbf{V}}^T\mathbf{P}\right)+\lambda {∥w∥}^2\right\}\mathrm{s}.\mathrm{t}.{\mathbf{1}}_N^{T}w=1.$$

(9)

3.3. Optimization Algorithm

The objective function Equation (9) is a multiple variables optimization problem, and it is non-convex with respect to w and $\mathbf{P}$ jointly. However, it is convex with either of them individually when the other is fixed. Thus, an alternative iteration strategy is adopted to get the solution of Equation (9). We first initialize w according to Equation (7). With w fixed, we optimize P according to Equation (8). The solution of Equation (8) is to find the eigenvectors corresponding to the first u largest eigenvalues of the matrix ${\left(\mathbf{VL}{\mathbf{V}}^T\right)}^{-1}\left(\mathbf{V}{\mathbf{D}}_v{\mathbf{V}}^T\right)$.

Next, fix P and optimize w:

$$\arg \underset{\mathit{w}}{\min }\left\{\mathrm{trace}\left({\mathbf{P}}^T\mathbf{VL}{\mathbf{V}}^T\mathbf{P}\right)+\lambda {∥\mathit{w}∥}^2\right\}\mathrm{s}.\mathrm{t}.{\mathbf{1}}_N^T\mathit{w}=1.$$

(10)

In this paper, we employ the Lagrangian algorithm to optimize the Equation (10). The Lagrangian function of the objective function (10) is defined as:

$$\begin{array}{cc}\hfill \psi \left(\mathit{w},c\right)& =\mathrm{trace}\left({\mathbf{P}}^T\mathbf{VL}{\mathbf{V}}^T\mathbf{P}\right)+\lambda {\mathit{w}}^T\mathit{w}+c\left({\mathbf{1}}_N^T\mathit{w}-1\right)\hfill \\ & =\frac{1}{2}\sum _{k=1}^N\sum _{i,j=1}^N\frac{w_k{h}_{ik}{h}_{jk}}{{\delta }_k}{∥{\mathbf{P}}^T{v}_i-{\mathbf{P}}^T{v}_j∥}_2^2+\lambda {\mathit{w}}^T\mathit{w}+c\left({\mathbf{1}}_N^T\mathit{w}-1\right).\hfill \end{array}$$

(11)

The partial derivatives of $\psi$ w.r.t. $w_i,i=1,2,\cdots ,M$ are given by:

$$\frac{\partial \psi \left(w,c\right)}{\partial w_k}=\frac{1}{2}\sum _{i,j=1}^N\frac{h_{ik}{h}_{jk}}{{\delta }_k}{∥{\mathbf{P}}^T{v}_i-{\mathbf{P}}^T{v}_j∥}_2^2+2\lambda w_k+c=0.$$

(12)

By simplifying Equation (12), $w_k$ can be calculated as:

$$w_k=-\frac{\frac{1}{2}{\displaystyle \sum _{i,j=1}^N}\frac{h_{ik}{h}_{jk}}{{\delta }_k}{∥{\mathbf{P}}^T{v}_i-{\mathbf{P}}^T{v}_j∥}_2^2+c}{2\lambda }.$$

(13)

According to the constraint $1_N^{T}w=1$, the Lagrange multiplier can be calculated as:

$$c=-\frac{1}{N}\left[\frac{1}{2}\sum _{k=1}^N\sum _{i,j=1}^N\frac{h_{ik}{h}_{jk}}{{\delta }_k}{∥{\mathbf{P}}^T{v}_i-{\mathbf{P}}^T{v}_j∥}_2^2+2\lambda \right].$$

(14)

By substituting Equation (14) into Equation (13), we can obtain $\mathit{w}$ finally.

Following this iteration process, w and P are alternately optimized until the maximal iteration number is reached or the relative difference of objective function value of Equation (9) is smaller than a given tolerance const $\epsilon$, i.e.,

$$\frac{\left|f(t+1)-f\left(t\right)\right|}{\left|f\left(t\right)\right|}⩽\epsilon ,$$

(15)

where $f(t+1)$ and $f\left(t\right)$ is the function value of Equation (9) at iteration $t+1$ and t, respectively. In addition, we can obtain the final projection matrix ${\mathbf{P}}^{*}$. At last, the joint feature set $\mathbf{V}$ is reduced as a low-dimensional feature set $\mathbf{Y}=\left[{\left({\mathbf{P}}^{*}\right)}^T{\mathit{v}}_1,\dots ,{\left({\mathbf{P}}^{*}\right)}^T{\mathit{v}}_N\right]$, which is then transmitted into an SVM classifier. Based on the above analysis, the proposed method can be summarized in Algorithm 1.

Algorithm 1: The proposed method ( denoted as SSHG*) for HSI classification.

Input: Data matrix $\mathbf{X}$, the reduced dimensionality u, the nearest neighbors number K and regularization parameter λ.     Output: The class-label vector $\mathit{f}$.   1  Normalize all the features to [0,1].   2  Build the MP from each of the PCs: $\mathbf{MP}\left(x\right)$ $=\{{\mathbf{CP}}_n\left(x\right),\dots ,\mathbf{I}\left(x\right),\dots ,{\mathbf{OP}}_n\left(x\right)\}$.   3  Obtain the EMP by stacking all MPs: $\mathbf{EMP}\left(x\right)=\{{\mathbf{MP}}_{{\mathrm{PC}}^1}\left(x\right),{\mathbf{MP}}_{{\mathrm{PC}}^2}\left(x\right),\dots ,{\mathbf{MP}}_{{\mathrm{PC}}^m}\left(x\right)\}.$   4  Represent the new stacked joint feature set as: $\mathbf{V}=[\mathbf{X};\mathbf{EMP}]=[{\mathit{v}}_1,{\mathit{v}}_2,\dots ,{\mathit{v}}_N]\in {\mathbf{R}}^{\left(m\right(2n+1)+d)\times N}.$   5  Compute the incidence matrix $\mathbf{H}\in {\mathbf{R}}^{\left|\mathbf{V}\right|\times \left|\mathbf{E}\right|}$ by KNN, set $\mathbf{H}(v,e)=1,\mathrm{if}v\in e$, otherwise, $\mathbf{H}(v,e)=0$.   6  Construct the hypergraph G and Calculate the weight of hyperedge $e_i$: $w\left(e_i\right)={\displaystyle \sum _{{\mathit{v}}_j\in e_i}}\exp \left(-\frac{{∥{\mathit{v}}_j-{\mathit{v}}_i∥}_2^2}{2{\sigma }^2}\right)$, the vertex degree: $d\left(v_j\right)={\displaystyle \sum _{e_i\in \mathbf{E}}}w\left(e_i\right)\mathbf{H}(v_j,e_i)$, and the hyperedge degree: $\delta \left(e_i\right)={\displaystyle \sum _{v_j\in \mathbf{V}}}\mathbf{H}(v_j,e_i)$.   7  Obtain the projection matrix $\mathbf{P}$ by optimizing Equation (8)   8  Solve Equation (10) and obtain the hyperedge weights computed as Equation (13).   9  With the new hyperedge weights, update ${\mathbf{D}}_v$, $\mathbf{L}$ and $\mathbf{W}$. 10  Repeat the steps 7–9 until the convergence criterion 15 is met or the maximal iteration number is reached. 11  Find the final projection matrix ${\mathbf{P}}^{*}$. 12  Project the joint feature set into a low-dimensional feature set: $\mathbf{Y}=\left[{\left({\mathbf{P}}^{*}\right)}^T{\mathit{v}}_1,\dots ,{\left({\mathbf{P}}^{*}\right)}^T{\mathit{v}}_N\right]$. 13  Feed the learned low-dimensional feature set $\mathbf{Y}$ into the SVM for classification. 14  return the class-label vector $\mathit{f}$.

4. Experiments and Discussion

4.1. Data Sets

In order to verify the performance of our proposed method, we conduct the experiments on the following three benchmark datasets.

(1)

Indian Pines data set—the first data set was acquired by the AVIRIS sensor over the Indian Pines test site in Northwestern Indiana, USA. The size of the image is 145 pixels × 145 pixels with a spatial resolution of 20 m per pixel. Twenty water absorption bands (104–108, 150–163, 220) were removed, and the 200-band image is used for experiments. Sixteen classes of interest are considered.

(2)

Pavia University data set—the second data set was acquired by the ROSIS sensor during a flight campaign over Pavia, northern Italy. The size of the image is 610 pixels × 340 pixels with a spatial resolution of 1.3 m per pixel. Twelve channels were removed due to noise. The remaining 103 spectral bands are processed. Nine classes of interest are considered.

(3)

Botswana data set—the third data set was acquired by the NASA EO-1 satellite over the Okavango Delta, Botswana, in 2001. The size of the image is 1476 pixels × 256 pixels with a spatial resolution of 30 m per pixel. Uncalibrated and noisy bands that cover water absorption features were removed, and the remaining 145 bands are used for experiment. Fourteen classes of interest are considered.

4.2. Experimental Setting

In order to demonstrate the effectiveness of adaptive weight estimation, we implement our algorithm as two versions. One is SSHG, which only utilizes the KNN hypergraph model for dimension reduction of the stacked feature set without adaptive weight estimation. The other is SSHG* shown in Algorithm 1. They are compared with the following feature extraction methods: (1) the method by using PCA to extract spectral features (denoted as PCA); (2) the method by using EMP features without dimension reduction (denoted as EMP); (3) the method [17] stacking the EMP and the spectral features as feature without dimension reduction (denoted as EMPSpe); and (4) the spatial hypergraph embedding method proposed in [36] (denoted as SH). In order to facilitate comparisons with these competing feature extraction methods, we adopt the overall accuracy (OA), the average accuracy (AA), the per-class accuracy and Kappa coefficient ($\kappa$) to evaluate the classification performance. Furthermore, the SVM classifier with Gaussian kernel is adopted to classify all of the aforementioned feature data of these feature extraction methods. The grid search tool is used to select the parameters of the optimal penalty term and Gaussian kernel variance in SVM within the given sets $\left\{2^{-10},...,2^{10}\right\}$ and $\left\{2^{-10},...,2^{10}\right\}$, respectively. The one-against-all strategy is adopted for multi-class classification. Regarding the three data sets, we select 15 samples from each class randomly to form a training set and the remaining samples are used as the test set. The training sample selection and the classification process are repeated ten times to reduce the bias induced by random sampling. We retain the average results. The parameters setting of SH is the same as the original paper [36]. With respect to our algorithm, the tolerance const $\epsilon$ is set as $1\times {10}^{-3}$ and the regularization parameter $\lambda$ is set as 100. The number of nearest neighbors K is selected as 10, 15, 5 for Indian Pines, Pavia University and Botswana data sets, respectively.

4.3. Experimental Results

The classification results of various methods upon three types of HSI are reported in

Table 1. Classification accuracy of various algorithms on the Indian Pines image.

Class	PCA (25)	EMP (27)	EMPSpe (227)	SH (22)	SSHG (44)	SSHG* (44)
1	91.61	98.71	99.03	94.87	98.06	98.06
2	47.36	61.46	64.28	82.59	72.53	73.96
3	48.60	78.75	77.14	73.50	84.06	84.85
4	68.29	95.90	91.76	91.32	96.76	97.21
5	75.75	87.78	88.85	92.12	89.83	90.32
6	85.37	91.48	92.36	98.22	93.93	94.04
7	91.54	99.23	99.23	100	100	100
8	79.52	98.47	98.92	98.31	99.57	99.63
9	96.00	100	100	100	100	100
10	56.22	74.23	71.61	87.51	76.81	77.68
11	49.62	69.51	71.02	64.41	75.65	75.57
12	45.43	75.67	77.40	84.31	84.33	84.79
13	93.47	98.68	99.00	99.49	99.37	99.37
14	69.55	93.25	94.83	94.84	97.57	97.58
15	46.42	95.96	95.85	75.07	97.74	97.76
16	89.62	97.56	98.46	98.75	99.74	99.87
OA	58.90	79.14	79.88	82.33	84.36	84.75
AA	70.90	88.54	88.73	89.71	91.62	91.92
kappa	53.88	76.42	77.24	80.06	82.27	82.73

,

Table 2. Classification accuracy of various algorithms on the Pavia university image.

Class	PCA (10)	EMP (27)	EMPSpe (130)	SH (30)	SSHG (46)	SSHG* (46)
1	66.21	82.40	81.57	70.33	81.67	82.70
2	65.14	83.44	84.09	82.13	92.02	91.44
3	70.00	77.04	77.79	72.37	80.47	80.08
4	85.26	97.42	97.44	89.58	93.93	94.90
5	99.37	99.76	99.75	99.61	99.79	99.80
6	69.16	78.91	80.16	91.76	86.50	89.63
7	90.45	94.07	93.28	92.68	94.16	94.44
8	71.34	86.12	85.30	72.16	83.07	84.06
9	99.72	96.04	97.44	99.51	98.26	98.15
OA	70.59	84.77	85.05	81.88	89.01	89.43
AA	79.63	88.35	88.53	85.57	89.99	90.58
kappa	63.20	80.38	80.78	76.80	85.64	86.24

and

Table 3. Classification accuracy of various algorithms on the Botswana image.

Class	PCA (22)	EMP (27)	EMPSpe (172)	SH (25)	SSHG (34)	SSHG* (34)
1	100	99.92	99.89	100	100	100
2	96.51	100	97.99	100	99.68	98.05
3	96.19	94.79	95.85	99.15	96.76	100
4	99.00	95.85	98.83	99.50	98.41	93.27
5	81.10	79.76	82.32	82.86	91.79	96.38
6	69.29	81.73	88.34	81.89	96.37	99.22
7	96.31	97.70	99.20	98.77	99.72	99.95
8	98.40	99.63	99.48	99.47	100	97.42
9	79.93	92.34	94.47	96.32	98.86	99.79
10	95.28	98.33	97.98	99.57	99.92	97.97
11	83.45	97.24	95.19	97.59	94.97	99.88
12	93.98	99.94	99.88	88.55	100	99.49
13	89.33	99.60	98.37	94.47	99.92	99.75
14	98.75	99.25	98.35	100	91.36	99.63
OA	89.83	94.69	95.65	95.10	97.79	98.38
AA	91.25	95.43	96.15	95.58	97.70	98.63
kappa	88.98	94.24	95.36	94.68	97.60	98.24

, respectively. The best results are highlighted with bold fonts. The number in brackets corresponds to the optimal dimensionality of reduced features. Classification maps of these different approaches are shown in Figure 3, Figure 4 and Figure 5, respectively. According to the experimental results, our proposed method achieves the highest OA, AA, and $\kappa$ among all of the competing methods, which shows the effectiveness of our feature extraction algorithm. The effectiveness of our SSHG method owes much to the hypergraph embedding of spatial and spectral joint features.

Comparing the EMP and EMPSpe method, we can find that EMPSpe method is always slightly better than EMP due to the fusion of EMP and spectral features for classification. As mentioned in [17], the stacked EMP and spectral features are transformed to low dimensional features by the decision boundary feature extraction (DBFE) and NWFE methods before classification. However, the DBFE and NWFE did not bring about the effective improvement of algorithm performance. SH utilized the hypergraph embedding model for feature reduction. Compared with PCA, the SH method has much better classification performance, which verifies the capacity of the hypergraph to capture the intrinsic complex relationships between HSI pixels. However, SH utilized only the spectral similarity for finding the nearest neighbors within a given spatial region. The superiority of SSHG over SH demonstrates that the embedding of EMP and spectral features is better for HSI classification. Specifically, our SSHG method can extract the rich spatial structures in the Pavia University data and achieve the maximum improvement upon this data. SSHG* obtains better classification results than SSHG, which demonstrates that adaptive hypergraph weight estimation is also beneficial for improving the classification accuracy.

There are two parameters, i.e., K and u, in our proposed method. The parameter K is the number of nearest neighbors, which determines how many pixels are included in the hyperedge. u is the dimensionality of the embedded low-dimensional feature. To evaluate their effects on the classification performance, we conduct the experiments on the above three datasets. We firstly fix the reduced dimensionality as $u=40$ and evaluate the influence of different K on the OA. As seen in Figure 6, when K is set as 10, 15, 5 for Indian Pines, Pavia University and Botswana data sets, respectively, the OA achieves the highest value. Taken as a whole, $[5,15]$ is usually a good range for the selection of parameter K. We then fix the K as 10, 15, 5 for the three datasets, respectively, and evaluate the influence of different us on the OA. Figure 7 shows the changes of OA with the reduced dimensions on three types of HSI. We can see that the inflection point of classification results is around the dimensionality 25 for these three HSIs, and there was no significant improvement on the classification results if the dimension continues to grow up.

5. Conclusions

In this paper, we propose a novel algorithm for spatial-spectral feature extraction based on hypergraph learning. A hypergraph is constructed by the KNN method and the embedding operation is conducted to transform the joint EMP and spectral features into the low-dimensional representation. Meanwhile, an efficient hypergraph weight estimation scheme is adopted to preserve the prominent hyperedges. Classification is performed with SVM using the embedded features. The experimental results on three benchmark hyperspectral datasets verify that our embedded representation can enhance the classification accuracy effectively. The hypergraph weight estimation can further improve the accuracy of HSI classification.