Next Article in Journal
Design Proposals for High-Voltage Stacked Configuration GaN Module
Previous Article in Journal
Learning Abstract Chemistry Concepts with Virtual Reality: An Experimental Study Using a VR Chemistry Lab and Molecule Simulation
Previous Article in Special Issue
Selective Grasping for Complex-Shaped Parts Using Topological Skeleton Extraction
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Multi-Scale Encoding Method with Spectral Shape Information for Hyperspectral Images

Key Laboratory of Exploration Technologies for Oil and Gas Resources, Ministry of Education, Yangtze University, Jingzhou 434025, China
*
Author to whom correspondence should be addressed.
Electronics 2024, 13(16), 3199; https://doi.org/10.3390/electronics13163199
Submission received: 11 July 2024 / Revised: 6 August 2024 / Accepted: 11 August 2024 / Published: 13 August 2024
(This article belongs to the Special Issue Applications of Machine Vision in Robotics)

Abstract

:
Spectral encoding is an important way of describing spectral features and patterns. Traditional methods focused on encoding the spectral amplitude information (SAI). Abundant spectral shape information (SSI) was wasted. In addition, traditional statistical encoding methods might only gain local adaptability since different objects should have their own best encoding scales. In order to obtain differential signals from hyperspectral images (HSI) for detecting ground objects correctly, a multi-scale encoding (MSE) method with SSI and two optimization strategies were proposed in this research. The proposed method concentrated on describing the SAI and SSI of the spectral reflectance signals. Four widely used open data sets were adopted to validate the performance of the proposed method. Experimental results indicated that the MSE method with SSI could describe the details of spectral signals accurately. It could obtain excellent performance for detecting similar objects with a small number of samples. In addition, the optimization strategies contributed to obtaining the best result from dynamic encoding scales.

Graphical Abstract

1. Introduction

Hyperspectral remote sensing technology collects the reflectance signals from the surface. It is important for large-scale earth observations. Hyperspectral imaging (HSI) can provide abundant spectral signals for detecting ground objects [1,2,3]. Spectral signal encoding methods are effective solutions for hyperspectral imaging because they can express spectral information and weaken noise influences [4,5,6]. In addition, spectral encoding methods can describe the spectral features directly. Detection methods based on the spectral code words can produce mapping results accurately compared to traditional spectral matching methods [7]. Therefore, research on spectral encoding methods plays an important role in applications utilizing HSI.
The existing spectral encoding methods adopted two ways to express spectral signals for detecting objects in HSI: encoding spectral amplitude information (SAI) and encoding spectral shape information (SSI) [8]. However, the majority of methods tended to only use the encoded SAI to recognize ground objects. For example, Jia and Richards used the binary coding method for fast spectral matching and classification [9]. Although it contributed to fast classification, it could not obtain excellent identification results because SAI encoding results were not enough to detect ground objects accurately, especially for similar objects. SSI was essential for translating spectral signals in detail. As a result, Chang et al., came up with a spectral derivative feature coding method to define SSI [10]. It provided an excellent solution for expressing spectral details. Many later methods used this strategy to describe SSI directly. Jiao et al., proposed a spectral DNA encoding method combining SAI and SSI for HSI classification [11,12]. This method could produce excellent imagery results because it used genetic algorithms and evolutionary computation to create and change the adjective code words for the matching and detecting processes. However, the original spectral DNA encoding method just encoded the SAI into only four kinds of code letters, namely, T, C, A, and G. In addition, it defined nine types of SSI into the four code letters as well. This meant that the detailed spectral information could not be mined from the original spectral DNA [13]. In 2016, based on the spectral DNA encoding method, a method of reference [8] was proposed to extract specific spectral DNA fragments for representing objects in hyperspectral images. It demonstrated exceptional classification performance. However, the spectral DNA fragments were obtained by an evolutionary computation strategy that was random. Thus, it was not suitable for describing hyperspectral stability. In 2022, Zhao and Tan proposed a spectral gene extraction (SGE) method based on the spectral DNA encoding method [14]. Specific spectral DNA fragments were extracted from the DNA-encoded spectra. It avoided the instability caused by evolutionary computation. The SGE method seemed proper for expressing hyperspectral images. However, since the SGE method came from the spectral DNA encoding method, it could only obtain rough spectral information.
Spectral signal encoding methods determined the mapping performances to a great extent [15,16]. Excellent spectral encoding methods must describe spectral information comprehensively [17]. Both SAI and SSI should be considered in the encoders to enhance identification abilities [18,19]. In addition, some spectral details should be hidden under the encoding results on different scales [20,21,22]. Static encoding methods, such as binary coding (BC) and the four-value coding (FVC) method, might only gain local performance [23,24,25,26]. Aiming at these problems, a multi-scale spectral encoding (MSE) method with SSI was proposed in this research for HSI. Firstly, it encoded the SAI and SSI into code sequences on dynamic scales. Then, the spectral code sequences were combined to describe the original spectral features. Finally, the objects in HSI were determined by matching the spectral code words. Two iteration strategies were established to obtain optimized results under the dynamic scales. The innovation of the proposed method was that SAI and SSI were both expressed in the proposed method. In addition, the MSE method with SSI translated SAI into dynamic scales. It used many code elements to translate the original spectral signals. Thus, the proposed method could mine more spectral details hidden in the original spectral data compared to traditional methods.
In this research, three experimental data sets were adopted to examine the performance of the MSE method with SSI. The experimental results indicated that the proposed method could obtain excellent performance in detecting all kinds of ground objects. According to the experiments and discussions, it was concluded from this research that: (1) the MSE method with SSI could describe spectral signal details comprehensively and accurately. (2) The proposed method contributed to object detection in HSI with small samples.

2. Materials and Methods

The proposed method consisted of the MSE method, the SSI encoding method, and the iteration optimization process. In the proposed method, the spectral amplitude and shape information of HSI were transferred into code word sequences by the encoding scales. Then, the best encoding scales were selected by the training and testing samples. Finally, the categories of HSI pixels were determined by matching the best encoded code words. The process of matching code words was optimized by an iteration strategy before producing the final results. In the optimization process, overall accuracy (OA) and each user’s accuracy (UA) were considered. In this research, four open and frequently-used hyperspectral datasets were used to validate the performance of the proposed method. The experimental datasets contained similar agricultural objects and urban objects. They were suitable for evaluating the image expression ability of the proposed method.

2.1. MSE Method

Encoding scales determined the description ability of the original spectral information. If the encoding scale was high, many code elements were used to describe the SAI. The code strands contained much more detailed information. In the limit, the code strands were very close to the original spectrum. If the encoding scale was low, bits of code elements were used to describe the SAI. The code strands contained a small amount of spectral information. In the limit, the method degenerated into a binary coding method.
T M = α × R i N
T j L H = M i n + j × T M R min 0.5 × E 1
T k U H = T M + k × R max T M 0.5 × E 1
Formulas (1)–(3) exhibited the calculations of the thresholds in the MSE method. TM was the middle threshold. Ri was the reflectance value of the i-th band. N was the number of bands. α was the coefficient of the middle threshold. TLH were the lower half thresholds and j was the index of lower thresholds. TUH were the upper half thresholds and k was the index of upper thresholds. Rmin and Rmax were the minimum and maximum reflectance values of the spectra, respectively. E was the number of the code element. According to the calculations, the original spectral signals of HSI and samples were encoded into code word sequences. If the amplitude coefficient (alpha in Formula (1)) was higher than 1.0, the upper half of spectral signals would be more detailed. On the contrary, the description of the lower half spectral signals was more detailed. Thus, by adjusting the amplitude coefficient, the MSE method could lean towards describing different spectral amplitude details.
Figure 1 shows the original hyperspectral signals and their multi-scale encoded results. The original hyperspectral signals were nearly continuous. When the encoding scale was two, the code strand was produced by two code elements: {0, 1} (Figure 1B). When the encoding scale was four, the code strand was produced by four code elements: {0, 1, 2, 3} (Figure 1C). The maximum encoding scale was thirty-six in this research: {0, 1, …9, A, B, …, Z}. Figure 1E,F showed the code strands produced by thirty-four and thirty-six code elements, respectively. It could be discovered intuitively that, with the encoding scale reducing, more code elements were used to describe the original hyperspectral signals. In addition, the code strands were more similar to the original hyperspectral signals.

2.2. SSI Encoding Method

Although the code strands produced by the MSE method could exhibit the SAI of the original spectral signals directly, the MSE method could not describe the SSI of spectral signals directly. It meant that the original spectral signals were not fully described by the MSE method. In order to describe spectral amplitude and shape information comprehensively, the SSI was defined as 9 types and encoded from 0 to 8, according to the research by Chang et al. Figure 2 shows the spectral shape definition and its code word.
T S = β × i = 1 N 1 ( R i + 1 R i ) N 1
Formula (4) was used to calculate the gradients of adjacent spectral signals when translating SSI. TS was the threshold of the spectral signal gradient. β was the coefficient of the spectral shape threshold. If the gradient of two adjacent spectral signals was less than TS, the spectral shape was regarded as flat. If the gradient of adjacent spectral signals surpassed TS, the spectral shape was regarded as rising or declining according to the size relationship between them. The spectral shape of every 3 adjacent spectral signals was encoded according to the spectral shape types (Figure 2). If the coefficient was higher than 1.0, it was helpful to filter tiny spectral waveforms caused by noise. If the spectral shape coefficient was lower than 1.0, small changes in spectral waveforms could be detected. It was beneficial for distinguishing categories with apparent spectral differences.
After being encoded, the SSI code word sequences were added to the end of SAI code words. In this way, the integrated spectral encoding results contained both spectral amplitude and shape information comprehensively. Using the integrated spectral code strands for feature matching and ground object detection was convenient and efficient. The process of using the MSE method with SSI and detecting targets in HSI is shown in Figure 3.

2.3. Iteration Optimization Process

Spectral code words could differentiate ground objects through matching calculations. However, different encoding scales led to different spectral code word sequences by the MSE method with SSI. Ground objects might have different best encoding scales to differentiate each other. Although the MSE method with SSI could describe the spectral information of ground objects well, the static encoding scale might not obtain the optimal global results. Thus, in this research, an iteration process was established to produce the optimized result (Figure 4).
Step 1: Encode the HSI into spectral code spaces by using the MSE method with the SSI according to scales.
Step 2: Encode all the samples uniformly and encode every category of samples, respectively, by the MSE method with SSI according to scales.
Step 3: Match the encoded samples with the encoded image and obtain a local result. For the unified encoding strategy, classification results could be produced directly. For the respective encoding strategy, detection results were produced from abundance results.
Step 4: Iterate the whole process according to the encoding scale. Every scale (corresponding to code elements) is set up once per iteration.
Step 5: Calculate Kappa, OA, and UA based on the classification results and test area.
Step 6: Create a classification result from the detection results by the respective encoding strategy according to the best UA in every category. Pick out the classification result from the classification result using a unified encoding strategy according to the best Kappa, resulting an output with a better result.
It should be noted that the best encoding scale for every category might be different. However, classification was an integrated process. An encoding scale that was best for one category did not mean that it was best for the others. Thus, two optimization strategies were provided in the iteration process of the MSE method with SSI to produce the optimal result: a unified encoding strategy (UES) and a respective encoding strategy (RES). For the UES, all the categories were encoded by the same code element numbers. The results with the best Kappa were produced as the final result. For the RES, every category must be encoded by diverse code element numbers. The temporary detection result for each category was used to calculate OA. The results with the best OA were gathered and produced the global result. After the iteration optimization process, the better results from UES and RES were output as the optimized result.

3. Results

In this research, four widely used HSIs were adopted to validate the performance of the proposed method. The MSE method with SSI needed only a number of samples to produce detection results. However, the methods based on Artificial Neural Net (ANN) needed many samples to train the network [27,28,29,30]. It was inequitable to compare the methods with the proposed method. Thus, we chose Minimum Euclidean Distance (MED), Spectral Angle Mapper (SAM), Binary Coding (BC), Spectral DNA matching (SDM), and Support Vector Machine (SVM) as contrast algorithms in this research. MED, SAM, and BC had no adaptive parameters. The α was 1 and the β was 1 of SDM. The kernel function of SVM was the radial basis function; the gamma in the kernel function was 0.02, and the penalty in the kernel function was 1.00. All the methods did not have a hyperparameter tuning process.

3.1. Experiment One

Experiment one was captured by the Airborne Visible Infrared Imaging Spectrometer (AVIRIS) in Salinas Valley, California, U.S. The spatial resolution of the data was 3.7 m. After removing water absorption bands (108–112, 154–167, and 224), 204 bands remained for detecting ground objects. The size of the experimental image was 241 × 241 pixels. There were eight kinds of objects contained in the experimental image: three kinds of fallow, celery, stubble, grapes, and two kinds of brocoli. Since the experimental data was open, reference data was also provided. The ground truth of the objects was marked in different colors.
Table 1 provides the training samples (TS) and ground truth (GT) information for experiment one. Figure 5A–H showed the false-color-composited image of experiment one, the ground truth, and the results produced by the contrast algorithms. For SVM, the gamma in the kernel function (radial basis function) was 0.005 and the penalty parameter was 100. The amplitude and shape coefficients of the MSE method with SSI were both 1.0 in the experiment. From the results, it was discovered that all the algorithms except BC could detect the labeled pixels in the experimental data. SDM performed poorly on Fallow_smooth, and SAM performed poorly on Grapes. SVM and MSE methods with SSI could detect the majority of ground objects correctly (Table 2). SVM produced some incorrect results for Brocoli_1 and Brocoli_2. The MSE method with SSI could distinguish them well because they had abundant differential code words (Figure 5J). However, the MSE method with SSI produced some incorrect results for Fallow_smooth. In general, the MSE method with SSI (E = 36) performed better than SVM in this experiment.
Figure 6 shows the encoding results of the samples in experiment one (α = 1.9, β = 1, and E = 12). Since the main ground objects in experiment one were vegetation and farmland, the spectra of them were very similar. However, the encoding results of the samples showed that the differences in the spectral code strands could be observed directly. It indicated that the MSE method with SSI could describe the spectral signals minutely. The spectral information description ability of the MSE method with SSI was important for detecting ground objects accurately. The MSE method with SSI obtained the best performance in this experiment.

3.2. Experiment Two

The data for this experiment was acquired in Longkou Town, Hubei Province, China, with a Headwall Nano-Hyperspec imaging sensor (Figure 7A). Table 3 lists the TS and GT of the experimental data. The size of the image was 550 × 400 pixels with 270 bands from 400 to 1000 nm. The spatial resolution of the image was about 0.463 m [31,32]. The referenced data was provided by the official website.
The detection results of experiment two are exhibited in Figure 7B–H. According to the field survey (Figure 7I), the farmlands were very wide, and some crops were in different growing stages. The spectral signals of the crops were very similar (Figure 7J). In addition, some bare soil existed in the study area. Thus, the detection results of the crops tended to be pixelated (E = 36, α = β = 1.0).
In the detection result for cotton, only the areas with white cotton were correctly detected. Some bared soil areas in the broad-leaf soybean were identified as other objects. However, most of the crops were identified correctly by the proposed method. By quantitative evaluation, the proposed method with 50~65 samples performed better than the contrast methods (Table 4). The narrow-leaf soybean, sesame, corn, and rice were detected with high accuracy. The image resolution in the experiment was very high (submeter level). It was a challenge for the target detection methods since it introduced more mixed pixels [33,34].

3.3. Experiment Three

The third dataset was urban data captured by the Hyperspectral Digital Imagery Collection Experiment (HYDICE) in October 1995 (Figure 8). The image area was located at Copperas Cove, near Fort Hood, Texas, U.S., with a size of 307 × 307 pixels. In this data, the band number was 210, the spectral resolution was 10 nm, and the spatial resolution was 2 m. Ground objects in the area included highways (asphalt roads), shopping malls, roads (concrete roads), grasses, trees, houses, and shadows.
The image for experiment three and the results produced by the contrast experiments are exhibited in Figure 8. The pixel number of training samples and ground truth were listed in Table 5. It could be observed intuitively that the results of MED and SAM were unsatisfactory. The result of MED misidentified asphalt roads from houses and the result of SAM misidentified asphalt roads from concrete road. The spectra of asphalt roads, concrete roads, and houses are shown in Figure 9. The spectral characteristics of them were amplified. It could be found that the spectral amplitude of asphalt roads and houses was very close. Thus, some pixels of asphalt road were misidentified as houses by MED. Although the spectral amplitude of asphalt road and concrete road was totally different, the waveforms of asphalt road and concrete road were nearly parallel. The spectral angles of their spectra were very similar. Thus, SAM could not distinguish asphalt road from concrete road well.
The result produced by SDM was excellent compared to the traditional spectral matching methods (MED, SAM, and BC). Its Kappa and OA were close to those of the SVM and the MSE methods with SSI (Table 6). However, the MSE method with SSI could express more spectral information about ground objects (E = 12). Thus, the result of the MSE method with SSI was more optimal compared to the result of SDM. In this experiment, the accuracy of SVM was slightly better (0.1%) than the proposed method in general. They both produced excellent detection results in experiment three. Compared to the results of experiments one and two, the performance of the proposed method in this experiment (downtown dataset) was not obvious. It might mean that the proposed method was more suitable for differentiating similar objects, such as crops.

3.4. Experiment Four

In this experiment, the hyperspectral image was captured by an airborne imaging spectrometer from the Xiaqiao test site (Figure 10). The size of the image was 346 × 400 pixels. The experimental data had 80 bands in spectral space from 417 to 854 nm. There were six kinds of ground objects in the image: road, corn, bare soil, vegetable, soil, and water.
The results produced by the contrast methods are exhibited in Figure 10. Table 7 lists the training samples and ground truth of this experiment. From the experimental results, it could be observed that BC performance was the poorest, because the soil and vegetable were not identified totally in the result of BC. It indicated that the BC method could not describe the spectral information of soil and vegetables in this experimental data effectively. MED, SDM, and SAM showed excellent performances. Associated with the result of SAM in experiment 1, it was discovered that SAM showed an instinctive ability to detect vegetation and plant covers. In addition, the detection accuracy of the MSE method with SSI (E = 4) was better than that of SVM in this experiment since SVM could not detect the vegetable in this experimental data well (Table 8).
During the experiments, the best code element number was determined by the HSI because the spectral differences of the ground objects were discrepant. In this research, the best code element numbers for experiments 1, 2, 3, and 4 were 36, 36, 12, and 4, respectively. When the code element number was two (binary coding with SSI), it never produced the best results. However, with the code element number increasing, the accuracy of the MSE method with SSI tended to be convergent. It made it possible to produce an optimal result. In addition, for the HSI with similar ground objects, such as experiment one and 2, the best code element number was large. As for the HSI whose spectral differences were sharp, such as experiment three and four, the best code element number was small.

4. Discussion

4.1. The Influence of Spectral Shape Encoding

Since the traditional spectral encoding method did not contain spectral shape information [35], detecting ground objects by the spectral encode method ignored spectral shape information. In order to describe spectral information comprehensively, the MSE method with SSI was established in this research. For ground objects with tiny spectral differences, MSE with SSI could obviously exhibit its advantage.
Figure 11A shows the spectral curves of houses and asphalt roads in experiment two. Figure 11B shows their spectral code strands using the traditional four-value encoding method. The encoding result only described the spectral amplitude information. Spectral shape information was vacant in the encoded result. Figure 11C shows their spectral code strands by MSE with an SSI of four code elements. It could be observed that the spectral amplitude of houses and asphalt roads was very close. Only 23 different spectral code words were found from their spectral code strands without SSI. However, when SSI was added to the spectral encoding process, 88 differential spectral code words were found in their spectral code strands. For these kinds of similar ground objects, MSE with SSI had much stronger identification abilities than the traditional spectral encoding method.
The detection results of the four-value encoding method and the MSE method with SSI are exhibited in Figure 12. The Kappa and OA of the four-value encoding method were 87.1% and 89.3%, respectively. The Kappa and OA of the MSE with SSI were 89.4% and 91.2%, respectively. Intuitively, the result of the four-value encoding method had more misidentified pixels of concrete roads and houses. It was consistent with the characteristics of the spectral code strands (Figure 11). However, the MSE method with SSI could distinguish concrete roads from houses better since it obtained more differential code words. As a result, MSE with SSI gained about a 2% accuracy improvement compared to the four-value encoding method. It meant that the spectral shape encoding method was important for differentiating ground objects with similar spectral signals. SSI helped the proposed method express more spectral details. Combined with the results of experiments one and two, it could be concluded that, although the training samples were limited, the MSE method with SSI could obtain excellent performance when detecting similar ground objects (agricultural crops).

4.2. The Influence of Encoding Strategy

In this research, the proposed MSE method with SSI had two kinds of dynamic optimization strategies: a unified encoding strategy (UES) and a respective encoding strategy (RES). Although both UES and RES must be used during the process of detection (Figure 4), the performance of the proposed optimization strategies needs to be discussed.
Every category of ground objects should have its own best encoding scale to distinguish it from the others. On this scale, its spectral code strands had the most unique spectral code words. That was the idea of the RES strategy. However, RES had shortcomings in practice. Figure 13 provides the abundance graphs of the categories produced by their best scales, which were calculated by RES. It could be discovered that the MSE method with SSI by RES could detect the target categories through their best scales. However, the overall detection result by RES was poor compared to the result by UES (Figure 14). Unidentified pixels (black points in Figure 14B) could be observed in the result by RES. That is probably because the result by UES was formed by the global optimal results of all the categories, but the result by RES was formed by the local optimal results of each category. The patchwork result (by RES) might lack collaboration between categories.
Although the detection accuracies of the MSE method with SSI by UES and RES were very close, the MSE method with SSI by UES gained higher Kappa and OA. In addition, the results produced by the MSE method with SSI by UES tended to be smoother and more artistic. The MSE method with UES was approximately 27.97–52.12% faster than the MSE method with RES. If the motivation of the task was to obtain the best recognition accuracy of the whole image, UES must be the proper strategy for the MSE method with SSI. However, if the motivation of the task was the detection of some specific objects, RES had excellent performance.

4.3. Coefficients of Spectral Amplitude and Shape

The spectral amplitude coefficient (α) and spectral shape coefficients (β) could influence the mapping accuracy. Especially for remote sensing identification methods involving crops [36,37]. However, it was hard to determine the coefficients before the detection process. In this research, we explored the rules and how they affected the detection performances of the experiments. The sensitivities of the spectral amplitude and shape coefficients were discussed in this section.
Table 9 lists the detection accuracies of experiment one by the proposed method through 12 code elements (not the best scale). The values of the coefficients ranged from 0.5 to 2.0, with a step length of 0.1. When one of the coefficients varied, the other coefficients stayed invariable. Figure 15 demonstrates the changes in accuracies affected by spectral shape and amplitude coefficients. It could be seen from Table 7 and Figure 15 that the performance of the proposed method improved when the spectral shape coefficient was 1.4 and the spectral amplitude coefficient was 1.9, compared to the situation when the coefficients were both 1.0. Bigger coefficients might be more suitable for this data using the MSE method with SSI.
The main misidentified objects in this data were brocoli_green_weeds_1 and broccoli_green_weeds_2. Figure 16 shows the spectral code strands produced by the different spectral amplitude coefficients. In the spectral code strand graphs, the red parts could represent the unshared code words. More unshared code words meant stronger differentiation ability. When the spectral amplitude coefficients were 1.0, 1.4, and 1.9, the unshared code words were 14, 15, and 19, respectively. It could be found that the number of differential code words increases with the increase in spectral amplitude coefficients. The unshared code words were the most common when the spectral amplitude coefficient was 1.9. The mapping results were consistent with the spectral encoding results. For the HSI with similar objects, greater spectral amplitude coefficients could help increase the detection performance. Proper coefficients could improve the accuracy of the proposed method. However, the OA and Kappa did not increase much while optimizing the parameters. It indicated that the detection accuracy was affected by the coefficients slightly. That meant the proposed method was resilient because the spectral details had been expressed by the MSE method with SSI. The detection performance did not rely on the adaptive parameters.

5. Conclusions

In this research, the MSE method with SSI and dynamic optimization strategies were proposed for describing hyperspectral images. Several conclusions were drawn from the experimental results: (1) SSI was important for the proposed method. It helped the MSE method with SSI initialize without a large set of samples. Small samples can make it perform well when detecting ground objects. (2) Encoding strategies suited for different conditions. RES was suitable for detecting specific ground objects. UES was suitable for obtaining the best global detection results. (3) The detection accuracy was affected by the coefficients slightly. It was resilient without a hyperparameter tuning process. However, it still contained many shortcomings compared to artificial intelligence (AI) algorithms. We will try to combine spectral encoding methods with AI.

Author Contributions

Conceptualization and methodology, D.Z.; validation, G.Z.; formal analysis and investigation, D.Z. and G.Z.; resources and data curation, D.Z.; data curation, G.Z.; writing—original draft preparation, D.Z.; writing—review and editing, G.Z.; visualization, D.Z.; supervision, G.Z.; project administration, D.Z.; funding acquisition, G.Z. All authors have read and agreed to the published version of the manuscript.

Funding

The APC was funded by the Open Fund of Key Laboratory of Exploration Technologies for Oil and Gas Resources (Yangtze University), Ministry of Education (K2021-6).

Data Availability Statement

Data are contained within the article.

Acknowledgments

The authors would like to thank Yanfei Zhong of Wuhan University for data support related to this work.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Chen, W.; Li, X.; He, H.; Wang, L. A review of fine-scale land use and land cover classification in open-pit mining areas by remote sensing techniques. Remote Sens. 2017, 10, 15. [Google Scholar] [CrossRef]
  2. Wang, Y.; Yu, W.; Fang, Z. Multiple kernel-based SVM classification of hyperspectral images by combining spectral, spatial, and semantic information. Remote Sens. 2020, 12, 120. [Google Scholar] [CrossRef]
  3. Zhou, G.; Chen, W.; Gui, Q.; Li, X.; Wang, L. Split depth-wise separable graph-convolution network for road extraction in complex environments from high-resolution remote-sensing images. IEEE Trans. Geosci. Remote Sens. 2021, 60, 1–15. [Google Scholar] [CrossRef]
  4. Li, W.; Feng, F.; Li, H.; Du, Q. Discriminant analysis-based dimension reduction for hyperspectral image classification: A Survey of the most recent advances and an experimental comparison of different techniques. IEEE Geosci. Remote Sens. Mag. 2018, 50, 7842–7851. [Google Scholar] [CrossRef]
  5. Xia, S.; Fu, F. On the average Hamming distance for binary codes. Discrete Appl. Math. 1998, 89, 269–276. [Google Scholar]
  6. Zhang, W.; Song, H.; He, X.; Huang, L.; Zhang, X.; Zheng, J.; Shen, W.; Hao, X.; Liu, X. Deeply learned broadband encoding stochastic hyperspectral imaging. Light Sci. Appl. 2021, 10, 108. [Google Scholar] [CrossRef] [PubMed]
  7. Mahlayeye, M.; Darvishzadeh, R.; Nelson, A. Cropping Patterns of Annual Crops: A Remote Sensing Review. Remote Sens. 2022, 14, 2404. [Google Scholar] [CrossRef]
  8. Wu, K.; Zhao, D.; Zhong, Y.; Du, Q. Multi-probe based artificial DNA encoding and matching classifier for hyperspectral remote sensing imagery. Remote Sens. 2016, 8, 645. [Google Scholar] [CrossRef]
  9. Jia, X.; Richards, J.A. Binary coding of imaging spectrometer data for fast spectral matching and classification. Remote Sens. Environ. 1993, 43, 47–53. [Google Scholar] [CrossRef]
  10. Chang, C.; Chakravarty, S. Spectral derivative feature coding for hyperspectral signature analysis. Proc. Int. Soc. Optical Eng. 2009, 42, 395–408. [Google Scholar] [CrossRef]
  11. Jiao, H.; Zhong, Y.; Zhang, L. Artificial DNA computing-based spectral encoding and matching algorithm for hyperspectral remote sensing data. IEEE Trans. Geosci. Remote Sens. 2012, 50, 4085–4104. [Google Scholar] [CrossRef]
  12. Jiao, H.; Zhong, Y.; Zhang, L. An unsupervised spectral matching classifier based on artificial DNA computing for hyperspectral remote sensing imagery. IEEE Trans. Geosci. Remote Sens. 2014, 52, 4524–4538. [Google Scholar] [CrossRef]
  13. Lv, N.; Han, Z.; Chen, C.; Feng, Y.; Su, T.; Goudos, S.; Wan, S. Encoding spectral-spatial features for hyperspectral image classification in the satellite internet of things system. Remote Sens. 2021, 13, 3561. [Google Scholar] [CrossRef]
  14. Zhao, D.; Tan, B.; Zhang, H.; Deng, R. Monitoring marine oil spills in hyperspectral and multispectral remote sensing data by the spectral gene extraction (SGE) method. Sustainability 2022, 14, 13696. [Google Scholar] [CrossRef]
  15. Cheng, X.; Fu, Z.; Yang, J. Improved multi-scale dynamic feature encoding network for image demoiréing. Pattern Recognit. 2021, 116, 107970. [Google Scholar] [CrossRef]
  16. Li, S.; Zhu, X.; Liu, Y.; Bao, J. Adaptive spatial-spectral feature learning for hyperspectral image classification. IEEE Access 2019, 7, 61534–61547. [Google Scholar] [CrossRef]
  17. Bhargava, R.; Falahaheirkhah, K. Enhancing hyperspectral imaging. Nat. Mach. Intell. 2021, 3, 279–280. [Google Scholar] [CrossRef]
  18. Park, S.; Lee, H.J.; Ro, Y.B. Adversarially robust hyperspectral image classification via random spectral sampling and spectral shape encoding. IEEE Access 2021, 9, 66791–66804. [Google Scholar] [CrossRef]
  19. Ma, C.; Lin, H.; Zhang, G.; Du, R. An efficient calibration method for multi-spectral imaging. Opt. Commun. 2018, 420, 14–25. [Google Scholar] [CrossRef]
  20. Lei, J.; Li, X.; Peng, B.; Fang, L.; Ling, N.; Huang, Q. Deep spatial-spectral subspace clustering for hyperspectral image. IEEE Trans. Circuits Syst. Video Technol. 2021, 31, 2686–2697. [Google Scholar] [CrossRef]
  21. Huang, J.; Shi, D.; Meng, W.; Zha, L.; Yuan, K.; Hu, S.; Wang, Y. Spectral encoded computational ghost imaging. Opt. Commun. 2020, 474, 126105. [Google Scholar] [CrossRef]
  22. Kang, X.; Zhang, A. A novel method for high-order residual quantization-based spectral binary coding. Spectrosc. Spectr. Anal. 2019, 39, 3013–3020. [Google Scholar]
  23. Song, K.; Yang, H.; Yin, Z. Multi-scale boosting feature encoding network for texture recognition. IEEE Trans. Circuits Syst. Video Technol. 2021, 31, 4269–4282. [Google Scholar] [CrossRef]
  24. Ma, X.; Liu, K.; Ding, C.; Yan, L.; Duan, M. Encoder-decoder with multi-scale information fusion for semantic image segmentation. In Proceedings of the ICGIP, Hangzhou, China, 3 January 2020. [Google Scholar]
  25. Li, Y.; Hu, B.; Gao, X.; Sun, X.; Wu, D.; Yan, P.; Zhang, X. Spectral data reconstruction algorithm of Hadamard transform spectral imager. Acta Photonica Sinica 2019, 48, 0630002-1-9. [Google Scholar]
  26. Ojala, T.; Pietikainen, M.; Maenpaa, T. Multi-resolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 2002, 24, 971–987. [Google Scholar] [CrossRef]
  27. Wang, Y.; Zhang, Y.; Song, H. A spectral-texture kernel-based classification method for hyperspectral images. Remote Sens. 2016, 8, 919. [Google Scholar] [CrossRef]
  28. Wang, Y.; Fang, Z.; Wang, M.; Peng, L.; Hong, H. Comparative study of landslide susceptibility mapping with different recurrent neural networks. Comput. Geosci. 2020, 138, 104445. [Google Scholar] [CrossRef]
  29. Tong, W.; Chen, W.; Han, W.; Li, X.; Wang, L. Channel-attention-based densenet network for remote sensing image scene classification. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2020, 13, 4121–4132. [Google Scholar] [CrossRef]
  30. Chen, W.; Li, X.; Wang, L. Fine land cover classification in an open pit mining area using optimized support vector machine and worldview-3 imagery. Remote Sens. 2019, 12, 82. [Google Scholar] [CrossRef]
  31. Zhong, Y.; Hu, X.; Luo, C.; Wang, X. WHU-Hi: UAV-borne hyperspectral with high spatial resolution (H2) benchmark datasets and classifier for precise crop identification based on deep convolutional neural network with CRF. Remote Sens. Environ. 2020, 250, 112012. [Google Scholar] [CrossRef]
  32. Zhong, Y.; Wang, X.; Xu, Y.; Wang, S. Mini-UAV-borne hyperspectral remote sensing: From observation and processing to applications. IEEE Geosci. Remote Sens. Mag. 2018, 6, 46–62. [Google Scholar] [CrossRef]
  33. Keshav, N.; Mustard, J.F. Spectral unmixing. IEEE Signal Process. Mag. 2002, 19, 44–57. [Google Scholar] [CrossRef]
  34. Janse, P.; Deshmukh, R. Crop Discrimination using Non-Imaging Hyperspectral Data. Int. J. Eng. Adv. Technol. 2021, 10, 269–273. [Google Scholar] [CrossRef]
  35. Varshney, P.K.; Arora, M.; Rao, R. Signal Processing for Hyperspectral Data. In Proceedings of the IEEE ICASSP, Toulouse, France, 24 July 2006. [Google Scholar]
  36. Yates, H.; Tarpley, J.; Schneider, S.; Mcginnis, D.; Scofield, R. The role of meteorological satellites in agricultural remote sensing. Remote Sens. Environ. 1984, 14, 219–233. [Google Scholar] [CrossRef]
  37. Li, L.; Huang, Z.; Zhou, J. A Two-Stage Maximum a Posterior Probability Method for Blind Identification of LDPC Codes. IEEE Signal Proc. Lett. 2020, 28, 111–115. [Google Scholar] [CrossRef]
Figure 1. Hyperspectral signals of a tree (A) and their multi-scale encoded results (BF).
Figure 1. Hyperspectral signals of a tree (A) and their multi-scale encoded results (BF).
Electronics 13 03199 g001
Figure 2. Spectral shape types and their code words.
Figure 2. Spectral shape types and their code words.
Electronics 13 03199 g002
Figure 3. Spectral encoding and mapping process by MSE method with SSI.
Figure 3. Spectral encoding and mapping process by MSE method with SSI.
Electronics 13 03199 g003
Figure 4. Iteration optimization process of the MSE method with SSI. H: height, W: width, B: bands, S: samples.
Figure 4. Iteration optimization process of the MSE method with SSI. H: height, W: width, B: bands, S: samples.
Electronics 13 03199 g004
Figure 5. Composited image for experiment one (A), ground truth (B), and the detection results ((C) MED, (D) SAM, (E): BC, (F) SDM, (G) SVM, (H) MSE methods with SSI, (I) spectra of the samples, and (J) spectral code words of brocolo_1 and brocolo_2).
Figure 5. Composited image for experiment one (A), ground truth (B), and the detection results ((C) MED, (D) SAM, (E): BC, (F) SDM, (G) SVM, (H) MSE methods with SSI, (I) spectra of the samples, and (J) spectral code words of brocolo_1 and brocolo_2).
Electronics 13 03199 g005
Figure 6. Spectral curves of the samples in experiment one and their spectral code strands formed by 12 code elements.
Figure 6. Spectral curves of the samples in experiment one and their spectral code strands formed by 12 code elements.
Electronics 13 03199 g006
Figure 7. Composited image for experiment two (A), ground truth (B), and the detection results ((C) MED, (D) SAM, (E) BC, (F) SDM, (G) SVM, (H) MSE methods with SSI, (I) photos of the samples, and (J) spectra of the samples. The colors of the spectral curves are consistent with the legend).
Figure 7. Composited image for experiment two (A), ground truth (B), and the detection results ((C) MED, (D) SAM, (E) BC, (F) SDM, (G) SVM, (H) MSE methods with SSI, (I) photos of the samples, and (J) spectra of the samples. The colors of the spectral curves are consistent with the legend).
Electronics 13 03199 g007
Figure 8. False-color-composite image for experiment three (A), ground truth (B), and the classification results ((C) MED, (D) SAM, (E) BC, (F) SDM, (G) SVM, (H) the MSE method with SSI).
Figure 8. False-color-composite image for experiment three (A), ground truth (B), and the classification results ((C) MED, (D) SAM, (E) BC, (F) SDM, (G) SVM, (H) the MSE method with SSI).
Electronics 13 03199 g008
Figure 9. Spectral curves of the asphalt road, concrete road, and house samples in experiment three and their spectral code strands formed by four code elements.
Figure 9. Spectral curves of the asphalt road, concrete road, and house samples in experiment three and their spectral code strands formed by four code elements.
Electronics 13 03199 g009
Figure 10. False-color-composited image of experiment three (A), ground truth (B), and the classification results ((C) MED, (D) SAM, (E) BC, (F) SDM, (G) SVM, (H) DMSE methods with SSI).
Figure 10. False-color-composited image of experiment three (A), ground truth (B), and the classification results ((C) MED, (D) SAM, (E) BC, (F) SDM, (G) SVM, (H) DMSE methods with SSI).
Electronics 13 03199 g010
Figure 11. Spectral curves of houses and asphalt roads (A) and their spectral code strands ((B) without SSI, (C) with SSI).
Figure 11. Spectral curves of houses and asphalt roads (A) and their spectral code strands ((B) without SSI, (C) with SSI).
Electronics 13 03199 g011
Figure 12. Results produced by the four-value encoding method (A) and the MSE method with SSI (B) using four code elements.
Figure 12. Results produced by the four-value encoding method (A) and the MSE method with SSI (B) using four code elements.
Electronics 13 03199 g012
Figure 13. Abundance graphs of the categories ((A) road, (B) corn, (C) bare soil, (D) vegetable, (E) soil, (F) water) in experimental data four by the MSE method with SSI through RES.
Figure 13. Abundance graphs of the categories ((A) road, (B) corn, (C) bare soil, (D) vegetable, (E) soil, (F) water) in experimental data four by the MSE method with SSI through RES.
Electronics 13 03199 g013
Figure 14. Results produced from experiment four by the MSE method with SSI through UES (A) and RES (B).
Figure 14. Results produced from experiment four by the MSE method with SSI through UES (A) and RES (B).
Electronics 13 03199 g014
Figure 15. Kappa and OA of the detection results in experiment one by the MSE method with SSI when spectral shape and amplitude coefficients varied. (A) accuracy curve produced by varied shape coefficients, (B) accuracy curve produced by varied brightness coefficients. The star represents the best accuracy.
Figure 15. Kappa and OA of the detection results in experiment one by the MSE method with SSI when spectral shape and amplitude coefficients varied. (A) accuracy curve produced by varied shape coefficients, (B) accuracy curve produced by varied brightness coefficients. The star represents the best accuracy.
Electronics 13 03199 g015
Figure 16. Spectral code strands of brocolis (AC) and classification results of experiment one using 12 code elements when spectral shape and amplitude coefficients were 1.0 and 1.0 (D), and 1.0 and 1.4 (E), 1.0 and 1.9 (F), respectively.
Figure 16. Spectral code strands of brocolis (AC) and classification results of experiment one using 12 code elements when spectral shape and amplitude coefficients were 1.0 and 1.0 (D), and 1.0 and 1.4 (E), 1.0 and 1.9 (F), respectively.
Electronics 13 03199 g016
Table 1. Ground objects and their samples for experiment one.
Table 1. Ground objects and their samples for experiment one.
Brocoli_1Brocoli_2FallowFallow_RoughFallow_SmoothStubbleCeleryGrapes
Training Sample1020202530302545
Ground truth32934371976139426783959357918,431
Table 2. Accuracy of the contrast algorithms in experiment one.
Table 2. Accuracy of the contrast algorithms in experiment one.
Value (%)MEDSAMBCSDMSVMMSE with SSI
Kappa94.293.684.495.297.197.5
OA95.995.589.196.798.098.3
Table 3. Ground objects and their samples for experiment two.
Table 3. Ground objects and their samples for experiment two.
CornBroad-Leaf
Soybean
SesameCottonNarrow-Leaf SoybeanRiceWaterRoad and HouseMixed Weed
Training Sample606560606060505050
Ground truth34,51163,21230318374415111,85467,05671245229
Table 4. Accuracy of the contrast algorithms in experiment two.
Table 4. Accuracy of the contrast algorithms in experiment two.
Value (%)MEDSAMBCSDMSVMMSE with SSI
Kappa58.670.267.874.382.683.0
OA56.770.268.574.582.783.2
Table 5. Ground objects and their samples for experiment three.
Table 5. Ground objects and their samples for experiment three.
Shopping MallTreesConcrete RoadHousesGrassesAsphalt RoadShadow
Training sample953946428411014
Ground truth2006159298569813261985245
Table 6. Accuracy of the contrast algorithms in experiment three.
Table 6. Accuracy of the contrast algorithms in experiment three.
Value (%)MEDSAMBCSDMSVMMSE with SSI
Kappa71.174.581.887.189.589.4
OA75.778.785.189.491.391.2
Table 7. Ground objects and their samples for experiment three.
Table 7. Ground objects and their samples for experiment three.
RoadCornBare SoilVegetableSoilWater
Training sample9539464284110
Ground truth125516213523851261589
Table 8. Accuracy of the contrast algorithms in experiment three.
Table 8. Accuracy of the contrast algorithms in experiment three.
Value (%)MEDSAMBCSDMSVMMSE with SSI
Kappa87.789.555.887.890.692.1
OA90.491.764.490.592.794.5
Table 9. Kappa and OA are calculated using different amplitude and shape coefficients.
Table 9. Kappa and OA are calculated using different amplitude and shape coefficients.
αβKappa(%)OA(%)αβKappa(%)OA(%)
1.00.596.097.20.5195.997.1
1.00.696.797.70.6197.298.1
1.00.796.797.70.7196.197.3
1.00.896.697.60.8197.298.1
1.00.996.497.50.9196.697.7
1.01.096.897.81.0196.897.8
1.01.196.597.51.1197.198.0
1.01.296.797.71.2197.398.1
1.01.396.797.71.3197.198.0
1.01.496.997.91.4197.198.0
1.01.596.897.81.5196.997.8
1.01.696.497.51.6197.198.0
1.01.796.397.41.7197.097.8
1.01.896.197.31.8196.597.6
1.01.996.797.71.9197.498.2
1.02.096.597.62.0197.198.0
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Zhao, D.; Zhang, G. Multi-Scale Encoding Method with Spectral Shape Information for Hyperspectral Images. Electronics 2024, 13, 3199. https://doi.org/10.3390/electronics13163199

AMA Style

Zhao D, Zhang G. Multi-Scale Encoding Method with Spectral Shape Information for Hyperspectral Images. Electronics. 2024; 13(16):3199. https://doi.org/10.3390/electronics13163199

Chicago/Turabian Style

Zhao, Dong, and Gong Zhang. 2024. "Multi-Scale Encoding Method with Spectral Shape Information for Hyperspectral Images" Electronics 13, no. 16: 3199. https://doi.org/10.3390/electronics13163199

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop