A Review of the Application of Hyperspectral Imaging Technology in Agricultural Crop Economics

Abstract

China is a large agricultural country, and the crop economy holds an important place in the national economy. The identification of crop diseases and pests, as well as the non-destructive classification of crops, has always been a challenge in agricultural development, hindering the rapid growth of the agricultural economy. Hyperspectral imaging technology combines imaging and spectral techniques, using hyperspectral cameras to acquire raw image data of crops. After correcting and preprocessing the raw image data to obtain the required spectral features, it becomes possible to achieve the rapid non-destructive detection of crop diseases and pests, as well as the non-destructive classification and identification of agricultural products. This paper first provides an overview of the current applications of hyperspectral imaging technology in crops both domestically and internationally. It then summarizes the methods of hyperspectral data acquisition and application scenarios. Subsequently, it organizes the processing of hyperspectral data for crop disease and pest detection and classification, deriving relevant preprocessing and analysis methods for hyperspectral data. Finally, it conducts a detailed analysis of classic cases using hyperspectral imaging technology for detecting crop diseases and pests and non-destructive classification, while also analyzing and summarizing the future development trends of hyperspectral imaging technology in agricultural production. The non-destructive rapid detection and classification technology of hyperspectral imaging can effectively select qualified crops and classify crops of different qualities, ensuring the quality of agricultural products. In conclusion, hyperspectral imaging technology can effectively serve the agricultural economy, making agricultural production more intelligent and holding significant importance for the development of agriculture in China.

1. Introduction

The development of agriculture, as the economic foundation of China, holds a significant position in the national economy. The yield and quality of crops receive considerable attention. High yields and good quality undoubtedly bring enormous benefits to China’s economy and the quality of people’s lives. The growth of crops from planting to maturity is a cyclical process during which crops may be affected by pests and diseases. Therefore, achieving non-destructive detection of crop pests and diseases is crucial. Additionally, the classification of agricultural products is a key step in ensuring their quality. How to achieve non-destructive classification and identification of agricultural products is a critical issue for ensuring quality.

In the above-mentioned issues, using traditional methods of detection and classification may lead to crop damage and low classification accuracy. Compared to traditional detection and classification methods, using hyperspectral imaging technology can effectively address this problem. Hyperspectral imaging technology mainly integrates multiple techniques to achieve detection and classification of targets. In agricultural development, it can realize non-destructive rapid detection of crops and improve the accuracy of crop product classification.

Hyperspectral imaging (HSI) technology originated from hyperspectral remote sensing technology and, together with laser radar technology and synthetic aperture radar technology, is known as one of the three core technologies for obtaining Earth observation information. The research on hyperspectral imaging technology was pioneered by the Jet Propulsion Laboratory (JPL) in the United States, starting in the 1980s [1]. In 1983, with strong support from the National Aeronautics and Space Administration (NASA), JPL developed the first airborne imaging spectrometer (AIS) internationally [2]. China’s research on hyperspectral imaging technology began slightly later, at the beginning of the 1990s [3]. The Shanghai Institute of Technical Physics was the first to successfully develop the airborne imaging spectrometer OMIS and the wide-field push-broom hyperspectral imaging instrument PHI [4].

The working principle of hyperspectral imaging technology is to utilize the differences in the reflection, absorption, and scattering characteristics of different materials to acquire rich spectral and spatial image information about the target object. The working process involves illuminating the target object with a light source, which generates a unique response in the form of reflection, scattering, or transmission of light. This response is then dispersed into different wavelengths by a spectral dispersion device, and separated and imaged by a grating, prism, or AOTF based on the principle of spectral separation. The detector collects the intensity data of different wavelengths, which are further processed to construct a hyperspectral image, enabling the in-depth analysis and research on the composition, structure, and state of the object. In contrast to traditional color or single-wavelength imaging methods, hyperspectral imaging can capture spectral information on the object’s reflection or emission over hundreds of continuous wavelengths. Moreover, hyperspectral imaging offers advantages such as providing richer information, enhancing identification and analysis capabilities, enabling non-destructive testing, and being applicable to a wide range of applications [5].

Hyperspectral imaging technology has been widely applied in various fields such as agriculture [6,7,8], environmental protection [9,10,11], medical diagnosis [12,13,14], and cultural heritage research [15,16,17] since its development. In agricultural production, it is commonly used for the non-destructive rapid detection of crop pests and diseases and the classification and identification of agricultural products. With the continuous development of this technology, the methods of acquiring original image data for detection objects using hyperspectral cameras have become more diversified and efficient, including indoor collection methods and UAV-based methods. The collected hyperspectral original image data are susceptible to environmental factors, equipment-related factors, and other influences, containing rich information and multidimensional data. Therefore, preprocessing is necessary before conducting data analysis. During data analysis, machine learning algorithms and deep learning algorithms are typically used to build recognition and classification models. After extensive data training, these models can effectively achieve the detection and identification of crop pests and diseases, as well as the classification of agricultural products.

In agricultural production, traditional methods for detecting and identifying crop diseases and pests are prone to crop damage, have low accuracy, and lack timeliness. Traditional methods for classifying and identifying agricultural products are time-consuming, labor-intensive, prone to product damage, and inefficient. Therefore, this article focuses on hyperspectral imaging technology, and combines common methods for hyperspectral data acquisition, data preprocessing, and data analysis to address the issues of detecting and identifying crop diseases and pests, as well as classifying and identifying agricultural products.

2. Development of Hyperspectral Imaging Applications

2.1. Crop Pest and Disease Detection and Identification Application Development

Since the successful implementation of hyperspectral imaging technology, it has gradually commercialized, leading to a large number of hyperspectral imaging instrument manufacturers emerging both domestically and internationally. The production of hyperspectral imaging instruments by these manufacturers is progressively applied in fields such as agriculture, forestry, and healthcare. Particularly in recent years, with the rapid development of agriculture and artificial intelligence, this technology combined with AI algorithms has been widely used in the agricultural production sector. It has found extensive applications in the detection of crop pests and diseases and the classification and identification of agricultural products.

Hyperspectral imagers can capture spectral characteristic changes in crops when affected by pests and diseases, enabling detection and early warning before or at the onset of pests and diseases [18]. This assists farmers in promptly implementing preventive measures to reduce losses caused by pests and diseases. By analyzing spectral information from different bands, this technology can identify and classify various types of pests and diseases, including viruses, bacteria, fungi, and insects [19]. This helps farmers accurately determine the type and severity of pests and diseases, providing a basis for precise control measures.

Researchers have conducted extensive studies using hyperspectral imaging technology to address the issue of crop pest and disease detection. By conducting detailed spectral sampling of crops, this technology can acquire continuous spectral curves of crops, analyze subtle spectral features, and thereby achieve pest and disease detection in crops.

Yu Jiajia et al. proposed the independent soft modeling method (SIMCA) for the early detection of gray mold on tomato leaves. They extracted characteristic spectral band images of gray mold on tomato leaves and fused the bands using multiple linear regression (MLR) to obtain a merged image, establishing a technical route for obtaining disease information using the minimum distance method. The results indicate that the proposed method exhibited excellent predictive capabilities, providing a new approach to early detection of gray mold on tomatoes [20].

Zhang Jingyi et al. aimed to detect Alternaria leaf spot disease in melons. They utilized hyperspectral imaging technology to acquire raw spectral images, preprocessed the raw data using principal component analysis and minimum noise fraction methods, and established leaf lesion discrimination models using the K-nearest neighbors and support vector machine methods. The study demonstrated the effectiveness of this method in detecting Alternaria leaf spot disease in melons [21].

Oilseed rape mycosphaerella is a soil-borne disease, with no visible symptoms on the leaves in the early stage of the disease, and it is difficult to detect from the surface of the plant. Liang Wanjie et al. used hyperspectral imaging technology combined with a deep learning model to construct a recognition model for the early onset of canola botrytis and achieved better recognition results [22].

Li Yang et al. sought to achieve the rapid online identification of cucumber diseases and pests. They used hyperspectral imaging and machine learning to study the rapid identification of cucumber downy mildew and two-spotted spider mite infestation. The model built using full-band spectral data preprocessed with the MA method showed the best identification performance, with the MA-RF model achieving an overall classification accuracy (OA) of 97.89% and a Kappa coefficient of 0.97. Research results indicated that using hyperspectral imaging technology for identifying cucumber downy mildew and two-spotted spider mite infestation yielded excellent results [23].

The above-mentioned study only identified a small number of samples, demonstrating the feasibility of using hyperspectral imaging technology in the detection of crop diseases and pests. However, due to the high cost and intricate operation of hyperspectral equipment, as well as the timeliness of detection, it is not feasible for large-scale use in agricultural production. With continuous technological innovation and the rapid development of IoT and unmanned aerial vehicle (UAV) technology, there is hope that hyperspectral imaging technology will be widely utilized in agricultural production.

In conclusion, the application of hyperspectral imaging technology in crop pest and disease detection can more efficiently assist farmers in managing their crops, promoting the rapid development of smart agriculture. This technology can make crop growth more intelligent and precise.

2.2. Development of Applications for Classification and Identification of Agricultural Products

Before the successful implementation of hyperspectral imaging technology, traditional methods were primarily utilized for agricultural product classification. These conventional techniques often require extensive manual intervention, are time-consuming, labor-intensive, and prone to errors. Hyperspectral imaging technology can capture multispectral information on the target, enabling the identification of distinct features of different agricultural products based on subtle spectral differences, thus achieving the precise and non-destructive classification of agricultural products [24].

Hyperspectral imaging technology enables rapid, non-contact detection of agricultural products, allowing for the identification of potential contaminants and harmful substances such as pesticide residues, thereby ensuring food safety [25]. Moreover, hyperspectral imaging technology plays a vital role in agricultural intelligence, facilitating the transformation of agricultural production methods and enhancing both the efficiency and quality of agricultural production.

Many researchers have effectively addressed the time-consuming, labor-intensive, and inefficient issues encountered in agricultural product classification and identification by utilizing hyperspectral imaging technology in conjunction with machine learning and deep learning methods. This technological approach enables the efficient and non-destructive identification of agricultural products.

Zhang Fu et al. proposed a support vector machine (SVM) classification model based on hyperspectral imaging technology can achieve the efficient and non-destructive identification of Ginkgo biloba varieties. They optimized model parameters using genetic algorithms (GA) and particle swarm optimization algorithms (PSO) to enhance the accuracy of variety identification. Experimental results indicated that the SNV-CARS-PSO-SVM model had the best identification performance, with a classification accuracy of 96.67%. This suggests that CARS feature wavelength variables can represent all wavelength information, and the PSO-SVM model demonstrates good variety identification performance, enabling the identification of Ginkgo biloba varieties [26].

Ma Lingkai et al. used hyperspectral imaging technology to differentiate between organic eggs and conventional eggs. They established egg category identification models using partial least squares discriminant analysis (PLS-DA) and support vector machines (SVM). Due to the large volume of hyperspectral data and the presence of significant redundant information, they applied the successive projection algorithm (SPA) and competitive adaptive reweighted sampling (CARS) to reduce the dimensionality of the egg yolk ROI data, eliminating redundant information to build the classification models. The experimental results showed that the SPA-SVM identification model achieved the highest accuracy rate of 94.2% on the test set [27].

Li Guohou et al. collected hyperspectral images of the front and back of wheat seeds from eight varieties using a hyperspectral imaging device. They proposed a hyperspectral wheat variety identification method based on an attention mechanism mixed with a convolutional neural network using this dataset. Experimental results demonstrated that the proposed method outperformed other methods with a classification accuracy of 97.92% [28].

Zhang Long et al. utilized hyperspectral imaging technology in combination with a Savitzky–Golay smoothing filter (SG), multiplicative scatter correction (MSC), and a support vector machine (SVM) to establish a classification method for tobacco leaves and contaminants. The experimental results indicated an overall classification accuracy of 99.92% for tobacco leaves and contaminants, with a Kappa coefficient of 0.998 [29].

In conclusion, the application of hyperspectral imaging technology in agricultural product classification and identification holds significant importance, as it can enhance classification accuracy, reduce manual costs, improve work efficiency, ensure food safety, and drive the development of intelligent agriculture.

3. Hyperspectral Data Acquisition

Nowadays, according to different application requirements in the field of agriculture, spectral data acquisition can be divided into two types: indoor acquisition and outdoor acquisition. The indoor acquisition system consists mainly of a hyperspectral camera, a halogen lamp, a camera obscura, and a slider. The advantage of this acquisition system is primarily the reduction in the impact of environmental factors, thereby improving the final recognition accuracy and classification precision. The lighting conditions in indoor environments may be influenced by artificial light sources, natural light, and the objects themselves, leading to variations in light intensity and color, which can affect the accuracy and stability of hyperspectral data. In indoor environments, objects may be obscured or affected by shadow effects, resulting in the occlusion or confusion in hyperspectral data, which can impact the data collection and analysis results. In order to achieve the classification and identification of sorghum varieties, Song Shaozhong et al. built an indoor spectral acquisition system based on hyperspectral imaging technology. By employing various preprocessing and data analysis methods, they achieved a classification accuracy of 97.16% [30]. The schematic diagram of the indoor acquisition system is shown in Figure 1.

Due to limitations in conditions, indoor acquisition is generally used for small-scale and limited sample testing. When there is a need to collect data on a large scale with multiple samples, outdoor acquisition methods are required. Outdoor acquisition methods mainly involve using drones equipped with hyperspectral cameras to collect spectral data. The outdoor acquisition system primarily consists of a drone, a gimbal, and a hyperspectral camera. The advantage of this method is the ability to collect data over large areas, which increases the collection rate. Outdoor data collection is susceptible to environmental factors such as lighting and weather, with lighting being one of the crucial environmental factors in hyperspectral data acquisition. Variations in lighting conditions can affect the reflectance spectral characteristics of objects, leading to color deviations or unstable reflectance rates in data collected under different lighting conditions. Weather factors such as overcast skies and cloudy conditions can also impact hyperspectral data collection. Under different weather conditions, factors like atmospheric water vapor content and aerosols can affect the transmission and scattering of light, thereby influencing the quality and accuracy of the data. Components in the atmosphere, such as gases and water vapor, can also affect the propagation of light and the reflection of objects. Atmospheric water vapor absorbs light at specific wavelengths, affecting the reflectance of specific bands in hyperspectral data. Factors such as lighting angles and vegetation coverage in different seasons can also impact hyperspectral data collection. Seasonal changes can alter the growth status and color of vegetation, thereby affecting the interpretation and analysis of the data. Feng Haikuan et al. utilized a drone equipped with a hyperspectral imager to monitor nitrogen levels in winter wheat [31].

Hyperspectral imaging is commonly performed using outdoor acquisition methods for detecting crop pests and diseases and monitoring their growth conditions. This method allows for the acquisition of a large amount of data, enabling the training of a significant dataset during data analysis, which can help reduce measurement errors. The outdoor data collection system primarily relies on data acquisition through unmanned aerial vehicles (UAVs). With continuous technological innovation, this collection system ensures real-time feedback on data acquisition information and collaborates with ground-based data analysis systems to facilitate the large-scale application of this data collection method in agricultural production.

4. Hyperspectral Data Processing

4.1. The Need for Hyperspectral Data Preprocessing

Hyperspectral raw data cannot be directly used for data analysis because various sources of noise may exist during the collection of the raw hyperspectral data of crops and agricultural products, such as atmospheric interference, instrument noise, and disturbances during the data acquisition process [32]. The hyperspectral raw data can also be influenced by factors like lighting conditions and sensor responses. By applying correction processes to the data, these influencing factors can be eliminated, making the data more accurate and comparable [33].

Hyperspectral raw data typically have high dimensions, containing a large amount of spectral information [34]. To enhance the efficiency of data processing and analysis, it is necessary to perform dimensionality reduction on the data, mapping the data from a high-dimensional space to a low-dimensional space while retaining the most representative information [35]. The raw hyperspectral data contain rich spectral information and extracting useful features from them is a critical issue. Feature extraction transforms the data into more representative and discriminative feature vectors, laying the foundation for subsequent modeling and analysis [36,37].

In summary, processing hyperspectral raw data involves noise removal, data correction, dimensionality reduction, and feature extraction. These preprocessing steps can enhance the quality and reliability of the data, reduce the impact of interfering factors, and provide a more reliable and effective basis for subsequent data analysis and applications, ultimately improving the accuracy of data analysis.

4.2. Hyperspectral Data Processing Flow

Hyperspectral raw data possess characteristics such as high dimensionality and diversity. Utilizing raw data directly for data analysis may lead to compromised detection results and classification accuracy. Therefore, it is crucial to process hyperspectral raw data with the main aims being to extract meaningful data, filter out irrelevant information, and eliminate various environmental influences [38]. The general data processing workflow is illustrated in Figure 2 below.

4.3. Hyperspectral Data Preprocessing Methods

4.3.1. Data Calibration and Pre-Processing

During the process of image data acquisition by a hyperspectral camera, the acquired images are raw and uncorrected, necessitating the post-acquisition correction of the image data. The primary reasons for this are as follows: on one hand, during image acquisition, the hyperspectral camera can be affected by dark currents and the surrounding environment, potentially impacting the stability of the acquired hyperspectral images [39]; on the other hand, since the raw hyperspectral image data consist of photon intensity information, they require reflectance correction to obtain relative reflectance. Therefore, black and white correction of hyperspectral raw data is a necessary step before data analysis [40,41]. Additionally, factors such as light scattering, irregularities in the detected object images, and random noise during spectral information acquisition can lead to issues like non-smooth spectral curves and a low signal-to-noise ratio. Hence, preprocessing of the data is typically conducted before any data analysis, with common preprocessing methods including smoothing, normalization, derivative methods, multi-scatter correction, standard normal variate transformation, wavelet transformation, etc., [42,43,44,45]. The data processed through these methods not only enhance curve smoothness and signal-to-noise ratio but also contribute to improved accuracy in subsequent modeling steps.

• Smoothing Processing

The spectral raw data collected are influenced by external factors such as noise and jitter, resulting in irregular and jagged spectral curves that are not conducive to later data analysis. After smoothing, the spectral curve not only eliminates the salient points in the original data but also makes the spectral curve more continuous and smoother. Common smoothing methods include median smoothing, mean smoothing, and the Savitzky–Golay (S–G) smoothing. The S–G smoothing filtering method [46] is a commonly used filtering technique in spectral preprocessing, initially proposed by Savitzky and Golay in 1964. This method is widely applied for data smoothing and noise reduction, utilizing local polynomial least squares fitting in the time domain [47,48]. Ma Yan et al. utilized different spectral preprocessing methods and support vector machine (SVM) techniques to achieve the identification of defects in fresh apricots. The research results indicate that the C-SVM model established using raw spectra and the Savitzky–Golay convolution smoothing preprocessing achieved the optimal recognition rate of 93.3% [49].

The core idea of the Savitzky–Golay (S–G) smoothing method is to perform weighted filtering on all data within a window, with the weights obtained through least squares fitting of a given high-order polynomial. The main feature of this filtering method is that it can remove noise while ensuring that the shape and width of the signal remain unchanged. The key to this smoothing method lies in finding appropriate parameters: the window width (N) and the order of the polynomial fit (K). The primary approach to implementing S–G smoothing is as follows: first, for the i-th reference point and its 2N + 1 neighboring points (N points on each side), conduct a least squares fit on the spectral data, obtaining polynomial coefficients (k). After solving for the coefficients a₀, a₁, …, a_k−1, replace the original intensity value of the reference point with the fitted value F_i [50].

F_i = a₀ + a₁x + … + a_k−1x_k−1,

(1)

where (X = −N, −N + 1, …, N). After iterating through the entire data band using this method, the smoothed spectral curve can be obtained, as shown in Figure 3 below.

• Normalization

In spectral analysis, the intensity values of spectral data may vary due to factors such as the concentration, temperature, and pressure of different samples. To eliminate the interference of these factors, it is necessary to normalize the spectral data for a more accurate comparison and analysis of the spectral characteristics of different samples [51]. The basic principle of spectral normalization is to standardize the intensity values of each wavelength point in the spectral data, ensuring that they fluctuate within a certain range without being influenced by external factors. Common normalization methods include Min-Max normalization, Z-score normalization, etc., [52,53]. Zhou Chunxin et al. achieved the non-destructive identification of soybean varieties using different preprocessing methods and data analysis techniques. The detection accuracy of the model built using normalized preprocessed spectral data reached 99.58% [54]. Min-Max normalization involves linearly transforming the values of each feature dimension so that they are mapped to the range [0, 1] (interval scaling), with the transformation function being as follows:

$$X^{\prime }={\displaystyle \frac{X-min}{max-min}},$$

(2)

The Z-score normalization method standardizes the values of each feature dimension using the mean and standard deviation of each dimension. Its transformation function is as follows:

$$X^{\prime }={\displaystyle \frac{X-\mu }{\sigma }},$$

(3)

• Derivative Method

The derivative method can reduce baseline drift caused by instrumental interference, uneven sample surfaces, lighting variations, and other factors. It partially addresses the issue of spectral signal overlap, amplifies hidden weak but effective spectral information, and enhances spectral changes and resolution [55]. The spectral derivative method is commonly used in the identification of absorption peaks and valleys in near-infrared spectra and in the extraction of characteristic wavelengths. Derivative spectra include first-order derivative spectra, second-order derivative spectra, and higher-order derivative spectra, with practical applications often requiring only first and second-order derivative spectra to meet requirements [56,57]. The application of first and second-order derivative processing on spectra is illustrated in Figure 4. Shen Yu et al. utilized the methods of loading coefficient, continuous projection, and second derivative for spectral data preprocessing to establish an apple minor damage identification model based on Genetic Algorithm-optimized BP Neural Network (GA-BP) and Support Vector Machine (SVM). The results showed that the recognition accuracy of the model established after processing with the second derivative method reached 99.69% [58].

• Multiple Scattering Correction

Multiple scattering correction (MSC) is a linearization process applied to spectra. Uneven sample distribution or variations in sample size can lead to light scattering, resulting in the inability to obtain an “ideal” spectrum [59]. This algorithm assumes that the actual spectrum is linearly related to the “ideal” spectrum. In most cases, the “ideal” spectrum is unattainable, so during spectral data preprocessing, the average spectrum of the sample data set is often used as a replacement [60]. Multiple scattering correction can eliminate random variations, and the corrected spectrum differs from the original spectrum. When the spectrum is highly correlated with the chemical properties of the substance being measured, multiple scattering correction yields better results [61,62]. He Jiawei et al. used near-infrared hyperspectral imaging technology to discriminate between fresh and frozen-thawed beef. After the multi-scattering correction preprocessing of the hyperspectral data, the optimal model for this experiment was constructed using partial least squares regression in the full spectral range (950–1500 nm). The experiment demonstrated that the model had a high prediction accuracy, with a discrimination rate of 94.4% [63]. In practical applications, it is common to use the average value of all data as the ideal spectrum/standard spectrum, which is expressed as follows:

$$\overline{S}={\displaystyle \frac{\sum _{i=1}^N{S}_i}{N}},$$

(4)

Next, perform a univariate linear regression on the spectrum of each sample with the standard spectrum to determine the baseline shift (k_i) and offset (b_i):

$$S_i=k_i\overline{S}+b_i,$$

(5)

The effect plot of the MSC correction is shown in Figure 5.

• Standard Normal Variate Transformation

Standard Normal Variate (SNV) transformation, similar to multiple scattering correction, can be used to eliminate scattering errors and path length variations [64]. However, the two methods have different processing principles. This method does not require an “ideal” spectrum; instead, it assumes that the spectral absorbance values at each wavelength in every spectrum satisfy certain conditions, such as following a normal distribution. The SNV transformation involves standardizing each spectrum [65,66]. Chen Shuyuan et al. proposed a non-destructive detection method for discriminating the storage years of white tea based on hyperspectral imaging technology. They utilized preprocessing algorithms including least squares smoothing filtering, standard normal transformation, normalization, and multi-scattering correction, and employed support vector machine, partial least squares, joint linear discriminant analysis, and logistic regression for the discriminant analysis of the spectral data after different preprocessing steps. The analysis results showed that the model established by combining standard normal transformation preprocessing with a support vector machine had the best discrimination effect, with precision rates of 90.83% and 86.02% for the training set and test set, respectively [67]. In practical processing, this method involves subtracting the original spectrum from the mean spectrum and then dividing by the standard deviation, effectively normalizing the original spectral data, satisfying the following equations:

$$Z_{ij}={\displaystyle \frac{x_i-\overline{x_i}}{S_{ij}}},$$

(6)

$$\overline{x_i}={\displaystyle \frac{\sum _{i=1}^N{x}_{ij}}{N}},$$

(7)

$$S_{ij}=\sqrt{{\displaystyle \frac{{\sum }_{i=1}^N{\left(x_{ij}-\overline{x_i}\right)}^2}{N-1}}},$$

(8)

where i is the number of samples, j is the number of spectral channels, and Z_ij is the SNV-transformed data. The effect plot of the SNV correction is shown in Figure 6.

• Wavelet Transform

Wavelet Transform (WT) is a newly developed time/frequency transformation analysis method used for spectral data compression and noise reduction [68]. Wavelet transform inherits the localized idea of Fourier transform, where the window size in the transformation varies with frequency. Moreover, when the time/frequency localization and multi-resolution properties are good, wavelet transform decomposes the spectral signal into multiple scale components at different frequencies. It selects the appropriate sampling step based on the size of the scale component, allowing it to focus on any spectral signal [69].

Wavelet transform typically includes Continuous Wavelet Transform (CWT) and Discrete Wavelet Transform (DWT) [70]. The multi-scale resolution characteristics of wavelet transform enable it to quickly extract the original spectral signal from noisy signals. Utilizing wavelet analysis can eliminate spectral signals with significant noise, making it a crucial application in hyperspectral analysis [71]. Huang Qi et al. conducted non-destructive rapid detection of peanut moisture content using hyperspectral imaging technology. They preprocessed the data using wavelet transform, multi-scattering correction, and first-order derivative, and established a non-destructive detection model for peanut moisture content using PLS, XGBoost, and BO-XGBoost algorithms. Through experimental comparisons, it was determined that the BO-XGBoost model established using spectral data preprocessed with wavelet transform from the original spectra was optimal [72].

• Basic formula for continuous wavelet transform

  $$W\left(a,b\right)={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}x\left(t\right)·\psi \left({\displaystyle \frac{t-b}{a}}\right)dt,$$

  (9)

  where W(a,b) is the wavelet coefficient, x(t) is the original signal, ψ(t) is the wavelet basis function, a is the scale parameter, and b is the shift parameter;
• Basic formula for discrete wavelet transform

  $$W\left(j,k\right)={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}x\left(t\right)·{\psi }_{j,k}\left(t\right)dt,$$

  (10)

  where W(j,k) is the discrete wavelet coefficient, and ψ_j,k(t) is the wavelet basis function.

4.3.2. Data Dimension Reduction and Feature Wavelength Extraction

Due to the high number of spectral bands, often hundreds or even thousands, in the raw data captured by hyperspectral cameras, there exists high-dimensional spectral information and significant data redundancy. This not only complicates the computation process but also diminishes the accuracy of non-destructive detection models [73]. Therefore, it is an important step in data analysis to perform dimensionality reduction and extract spectral feature wavelengths from hyperspectral raw data before modeling. The most commonly used methods for dimensionality reduction and feature wavelength extraction include principal component analysis, independent component analysis, genetic algorithms, and minimum noise fraction transformation [74,75]. After processing with the corresponding dimensionality reduction algorithms, a significant amount of redundant information is eliminated. By extracting feature wavelengths, the most distinctive and information-rich wavelengths can be selected to better understand the data and perform target identification or classification [76]. These processes play a crucial role in simplifying the computation process and enhancing the accuracy of the models.

• Principal Component Analysis

Principal Component Analysis (PCA) is a commonly used dimensionality reduction method in statistics and mathematics. It transforms multiple possibly correlated variables into a set of linearly uncorrelated variables through orthogonal transformation, known as principal components. It aims to convert high-dimensional features into a few key components that are mutually uncorrelated and contain the maximum amount of information [77].

The basic idea behind PCA is to recombine several originally correlated indicators into a new set of composite indicators that are mutually independent, while retaining as much information from the original variables as possible [78]. This method is highly effective in handling multivariable problems as it reduces data dimensionality, decreases computational complexity, and eliminates noise and irrelevant features, making the dataset more user-friendly and interpretable. The computational steps of PCA typically involve standardizing the samples, calculating the covariance matrix of the standardized samples, determining the eigenvalues and eigenvectors of the covariance matrix, computing the contribution rate and cumulative contribution rate of the principal components, and selecting the principal components that achieve a certain level of cumulative contribution rate [79]. These principal components can be utilized for subsequent data analyses such as cluster analysis, regression analysis, etc., [80].

For hyperspectral data represented as an i×j matrix:

$$X=\left(\begin{array}{ccc}{x}_{11}& \cdots & x_{1j}\\ ⋮& \ddots & ⋮\\ x_{i1}& \cdots & x_{ij}\end{array}\right)=\left(x_1,x_2\cdots x_j\right),$$

(11)

where i is the number of samples, and j is the number of spectral channels. The main steps of the PCA method are as follows:

• Standardize the matrix X with a mean of $\overline{x}$ and a standard deviation of S_n

  $$X_{ij}={\displaystyle \frac{x_{ij}-\overline{x}}{S_n}},$$

  (12)

  whereby the original matrix is standardized to obtain matrix X1 = (X₁, …, X_j);
• Calculate the correlation matrix along with its eigenvalues and eigenvectors;
• Compute the variance contribution rates and determine the principal components. Typically, select the top k principal components whose cumulative contribution rate exceeds 90% for data analysis.

• Independent Component Analysis

Independent Component Analysis (ICA) is a statistical method used for separating multivariate signals, aimed at decomposing multiple mixed signals into independent components [81]. Unlike other linear transformation methods (such as PCA), ICA assumes that the original signals are mutually independent rather than simply having low correlation. Its core idea is based on the assumption of independence, positing that the observed multivariate signals are linear mixtures of multiple independent components, each with its own statistical distribution [82]. The assumption of independence is crucial to ICA, enabling the decomposition of mixed signals into independent components through appropriate transformations. This algorithm is currently widely applied in fields such as signal processing, image processing, and other domains [83].

Assuming that the observed mixed signal X is a linear combination of unknown independent signals S, satisfying:

$$X=AS,$$

(13)

in this case, X is the observed signal matrix, A is the unknown mixing matrix, and S is the source signal matrix.

The goal of ICA is to estimate the inverse or pseudo-inverse of the mixing matrix A in order to recover the source signal S. The main steps of ICA are:

• The purpose is to subtract the mean of each feature (variable) of the data to ensure that the data have a mean of zero on each dimension

  $$X_{centered}=X-E\left[X\right],$$

  (14)

  where E[X] is the mean of each column (feature) in the original data matrix X;
• The aim is to transform the observed data into a new space with a covariance matrix that is the identity matrix, reducing data redundancy and speeding up the convergence of ICA. Whitening involves transforming the centered data into a new coordinate system such that the transformed signals are uncorrelated in each direction and have unit variance. Whitening can be achieved by computing the covariance matrix C of the data and then performing eigendecomposition to accomplish this

  $$C={\displaystyle \frac{1}{n-1}}{X}_{centered}{X}_{centered}^T,$$

  (15)

  $$C=ED{E}^T,$$

  (16)

  where E is the matrix of eigenvectors, and D is the diagonal matrix of eigenvalues. The whitening transformation is as follows:

  $$X_{white}=D^{-1/2}{E}^T{X}_{centered},$$

  (17)
• Select a measure of non-Gaussianity (such as negative entropy) and find the projection weights that maximize this measure. The ICA algorithm typically uses a nonlinear function g (∙) to approximate the maximization of negative entropy, with the iterative update rule as follows

  $$w^{+}=E\left[X_{white}g\left(w^T{X}_{white}\right)\right]-E\left[g^{\prime }\left(w^T\right)X_{white}\right]w,$$

  (18)

  normalize the new weight vector w⁺:

  $$w={\displaystyle \frac{w^{+}}{\parallel w^{+}\parallel }},$$

  (19)
• In multidimensional scenarios, it is necessary to ensure that the estimated components are orthogonal to each other. The Gram–Schmidt process is commonly employed for this purpose. For the i-th component, the update rule takes into account the previous i−1 components

  $$w_i=w_i-{\sum }_{j=1}^{i-1}{w}^T{w}_j{w}_j,$$

  (20)

  and after normalizing w_i, check for convergence. If the difference between w_i and $w_i^{+}$ is less than a certain threshold, it is considered that an independent component has been found.
• Once all the weight vectors W = [ w₁, w₂, …, w_n] are found, independent components can be estimated as follows

  $$S=W^T{X}_{white},$$

  (21)

• Main Steps of MNF

Minimum Noise Fraction (MNF) is a method used to determine the intrinsic dimensionality (i.e., number of spectral bands) of image data [84]. This method can separate noise in the data, reducing computational demands in subsequent processing. MNF is essentially a two-stage stacked principal component transformation. The first transformation (based on the estimated noise covariance matrix) is used to separate and readjust noise in the data, ensuring that the transformed noise data have minimal variance and no inter-band correlation. The second step involves a standard principal component transformation of noise-whitened data. To further conduct spectral processing, the intrinsic dimension of the data is determined by examining the final eigenvalues and associated images [85]. The data space can be divided into two parts: one part is related to larger eigenvalues and the corresponding feature images, while the remaining part is related to approximately equal eigenvalues and images dominated by noise [86,87]. Main steps of MNF:

• Utilize a high-pass filter template to filter the entire image or image data blocks with the same characteristics, obtaining the noise covariance matrix (C_N). Diagonalize it into a matrix (D_N)

  $$D_N=U^T{C}_{N}U,$$

  (22)

  where D_N is the diagonal matrix of eigenvalues of C_N arranged in descending order; U is an orthogonal matrix composed of eigenvectors. Further transformation formulas can be derived as follows:

  $$I=P^T{C}_{N}P,$$

  (23)

  $$P=U{D}_N^{-1/2},$$

  (24)

  where I is the identity matrix, P is the transformation matrix. When P is applied to image data X through the transformation Y = PX, the original image is projected into a new space, generating transformed data where the noise has unit variance and is uncorrelated between bands.
• Conduct standard principal component transformation on the noise data with the formula

  $$C_{D-adj}=P^T{C}_{D}P,$$

  (25)

  where C_D is the covariance matrix of image X. C_D-adj is the matrix after transformation by P, further diagonalized into matrix D_D-adj.

  $$D_{D-adj}=V^T{C}_{D-adj}V,$$

  (26)

  where D_D-adj is a diagonal matrix of eigenvalues of C_D-adj arranged in descending order; V is an orthogonal matrix composed of eigenvectors.

After the MNF transformation, the resulting components are uncorrelated with each other, arranged in a descending order of signal-to-noise ratio. MNF transformation separates the noise and ensures inter-band decorrelation, making it superior to PCA transformation. Currently, it is widely used in image denoising, image fusion, image enhancement, and feature extraction, among other applications.

• Genetic Algorithm

The Genetic Algorithm (GA) is an optimization algorithm that simulates the evolutionary process in nature. By mimicking genetic, crossover, and mutation operations in biology, it progressively searches for the optimal solution [88]. The principle of genetic algorithm dimensionality reduction is an effective method of high-dimensional data analysis. Through the simulation of the biological evolution process in nature it extracts the most representative combination of features from high-dimensional data to achieve the optimization of data reduction and peacekeeping [89]. The core idea of genetic algorithm dimensionality reduction is to select the most representative features from high-dimensional data through the optimization process of genetic algorithms and reduce the high-dimensional data to a lower-dimensional space [90]. The main stages of the genetic algorithm process are illustrated in Figure 7. The specific steps are as follows:

• Generate a set of solutions with random feature combinations;
• Evaluate the quality of each solution using an evaluation function such as information gain, variance, etc.;
• Choose excellent solutions based on fitness evaluation results as parents for the next generation of the population;
• Exchange and recombine chromosomes in parents, such as single-point crossover or multi-point crossover;
• Introduce randomness by mutating some solutions to increase population diversity, like bit flips or bit swaps;
• Determine which solutions to retain based on replacement strategies like elitism or randomness;
• Set stopping conditions for the algorithm, such as maximum number of iterations or fitness function reaching a threshold value.

Data dimensionality reduction plays a crucial role in data processing, and researchers have proposed various methods for data dimensionality reduction over the years. Wu Qiang et al. introduced a convolution-based spectral data machine learning feature dimensionality reduction method. Compared to the data method of principal component analysis, this method showed significantly better classification performance on theoretical data [91]. Feng Linlin et al. proposed a data dimensionality reduction method based on variational autoencoder uniform manifold approximation and projection. To reduce the coupling between high-dimensional data, the variational autoencoder was used to preprocess the data into latent variables, which were further reduced using uniform manifold approximation and projection to better maintain the similarity relationships between the original high-dimensional data [92].

Currently, data dimensionality reduction can be classified into two types: linear data dimensionality reduction and nonlinear data dimensionality reduction. In addition to the described dimensionality reduction methods, there are many algorithms suitable for data dimensionality reduction processing. Different data dimensionality reduction methods can be chosen based on the characteristics of the data. Commonly used linear data dimensionality reduction methods include Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), Multidimensional Scaling (MDS), Margin Discrete Projection (MDP), and Linear Local Tangent Space Alignment (LLTSA). Nonlinear data dimensionality reduction methods include Local Linear Embedding (LLE), Locally Preserving Projections (LPP), and Isometric Mapping (ISOMAP), among others.

4.4. Hyperspectral Data Analysis Methods

4.4.1. Machine Learning Methods

Machine learning methods play a crucial role in the analysis of hyperspectral data. After data preprocessing is completed, machine learning methods are primarily used to model and analyze the data, thereby achieving the detection and identification of crop diseases and pests, as well as the classification and identification of agricultural products. Commonly used machine learning methods in different application scenarios include Support Vector Machine (SVM), Random Forest (RF), K-Nearest Neighbors (KNN), and Linear Discriminant Analysis (LDA).

• Support Vector Machine

Support Vector Machine (SVM) is a commonly used supervised learning algorithm that is applicable to both binary and multiclass problems [93]. Its principle is based on the minimization of structural risk in statistical learning theory by finding an optimal hyperplane for classification. The basic principle of SVM is as follows:

• Assuming the training dataset is linearly separable, meaning there exists a hyperplane that can completely separate samples from different classes. The hyperplane can be represented by the following equation:

  $$w·x+b=0,$$

  (27)

  where w is the normal vector, x is the sample feature, and b is the bias term;
• The goal of SVM is to find an optimal hyperplane that maximizes the margin between the sample points and the hyperplane. The margin refers to the distance from the sample points to the hyperplane, and maximizing this margin helps improve the generalization ability of SVM [94]. The margin can be expressed as

  $${\gamma }_i=y_i\left({\displaystyle \frac{w}{\parallel w\parallel }}{x}_i+{\displaystyle \frac{b}{\parallel w\parallel }}\right),$$

  (28)

  where y_i is the label of the sample (1 or −1) and ||w|| is the norm of the normal vector w;
• During the process of maximizing the margin, only a subset of sample points has an impact on the position of the hyperplane, and these points are called support vectors. Support vectors are the sample points closest to the hyperplane, determining the position and orientation of the hyperplane;
• Transforming the goal of maximizing the geometric margin into a convex optimization problem can be solved using the Lagrange multiplier method. By maximizing the margin, the optimization problem can be reformulated in its dual form, where the objective is to minimize a function involving Lagrange multipliers;
• In practical applications, many problems cannot be perfectly separated by a linear hyperplane. To address this issue, SVM introduces the concept of kernel functions, mapping nonlinear problems from low-dimensional spaces to high-dimensional spaces to make them linearly separable. Common kernel functions include linear kernel, polynomial kernel, Gaussian kernel, etc., [95];
• In real-world scenarios, data often contain noise and outliers. To enhance the robustness of the model, SVM introduces the concepts of soft margin and penalty terms. Soft margin allows some sample points to fall within the margin by introducing slack variables to control this. The penalty term balances maximizing the margin and penalizing misclassifications through regularization [96];
• For multi-class problems, one can use the one-vs-one or one-vs-rest methods to con-struct multiple binary classifiers, then combine them for multiclass classification.

• Random Forest

The Random Forest (RF) algorithm is a classification and regression method based on ensemble learning [97]. Its core idea is to improve prediction performance by constructing multiple decision trees and combining them. Specifically, when building each decision tree, the random forest algorithm uses random sampling to extract samples from the original dataset, and randomly selects a subset of features at each node to split the decision tree [98]. This randomness helps reduce the correlation between models, increase model diversity, prevent overfitting, and enhance generalization capability. The main steps of RF are as follows:

First, assuming there are T samples and M features available for a classification task, each sample is associated with a corresponding label yi. Next, n decision trees are built, where each tree is constructed on a randomly selected subset of samples, considering only a subset of all features during the tree-building process [99]. Each feature is referred to as a predictor variable, and all predictor variables can be denoted as {X₁, X₂, …, X_m}. For each tree, the random forest algorithm defines the following steps:

• Sample training set samples from the sample set using sampling with replacement;
• Randomly select k variables without replacement from all predictor variables, where k is much less than m. The selected set of variables can be represented as {f₁, f₂, …, f_k};
• Build a decision tree based on the training samples and the generated subset of variables, using a certain criterion to measure the importance of features to determine the feature for the first split node in the tree;
• Repeat step (3) n times to generate n decision trees.

An example diagram is shown in Figure 8.

• K-Nearest Neighbors Algorithm

The K-Nearest Neighbors (KNN) algorithm is a classification method based on Euclidean distance to infer the category of items. It is used to accomplish hyperspectral classification by computing distances between two targets [100,101].

The distance calculation method is as follows:

$$D\left(x,y\right)=\sqrt{{\sum }_{k=1}^n{\left(x_k-y_k\right)}^2},$$

(29)

where x and y represent two targets, x_k and y_k denote the coordinates of the two targets in the k-th dimension, and n is the dimension of the target coordinates. The main implementation steps of the K-nearest neighbors algorithm are as follows:

• Collect and prepare a dataset for training and testing. Each sample should include a set of features and corresponding labels (for classification) or target values (for regression);
• Determine the value of K, which represents the number of nearest neighbor samples to consider. The choice of the K value affects the algorithm’s performance and is typically selected through methods like cross-validation;
• For a given test sample, calculate the distance between it and each sample in the training set. Common distance metrics include Euclidean distance, Manhattan distance, etc.;
• Based on the calculated distances, select the K nearest training samples to the test sample as its nearest neighbors;
• For classification problems, determine the category of the test sample using a majority voting approach. Predict based on the most frequently occurring category among the labels of the K nearest neighbors. For regression problems, average the target values of the K nearest neighbors to obtain the predicted value for the test sample;
• Based on the results of the majority voting or averaging operation, consider it as the final prediction result for the test sample.

When selecting the value of K, it is essential to balance the sensitivity and complexity of the model. A small K value is susceptible to noise interference, leading to underfitting and missing data features. On the other hand, a large K value may result in over-smoothing, potentially causing overfitting and reduced generalization. In imbalanced data, an inappropriate K value can bias towards the majority class. The appropriate K value affects the smoothness of boundaries; a smaller K value results in complexity and high variability, while a larger K value leads to smoother results but may lose detail [102].

• Linear Discriminant Analysis

Linear Discriminant Analysis (LDA) is a supervised classification method, which requires prior knowledge of the sample classification results in the training set. This method linearly transforms high-dimensional feature vectors into low-dimensional feature vectors. This transformation ensures that the data features of samples belonging to the same class are close to each other, while the features of samples from different classes are significantly different [103,104].

LDA primarily constructs a low-dimensional “boundary” to project each sample, such that after projection, the within-class variance is minimized and the between-class variance is maximized [105]. This boundary is obtained under supervised classification samples and can be used for classifying test sets and unknown samples. The basic steps of LDA are as follows:

• Compute the within-class scatter matrix and between-class scatter matrix: The within-class scatter matrix (S_W) represents the dispersion of data points within the same class, while the between-class scatter matrix (S_B) represents the dispersion between different classes;

  $$S_W={\sum }_{i=1}^c{\sum }_{x\in D_i}\left(x-m_i\right){\left(x-m_i\right)}^T,$$

  (30)

  where c is the number of classes, D_i is the dataset of the ith class, and m_i is the mean vector of the ith class.

  $$S_B={\sum }_{i=1}^c{N}_i\left(m_i-m\right){\left(m_i-m\right)}^T,$$

  (31)

  where N_i is the number of samples in the ith class, and m is the global mean vector of all samples;
• Calculate the eigenvalues and eigenvectors of matrix ${\mathrm{S}}_{\mathrm{W}}^{-1}{\mathrm{S}}_{\mathrm{B}}$: The eigenvalues of matrix ${\mathrm{S}}_{\mathrm{W}}^{-1}{\mathrm{S}}_{\mathrm{B}}$ represent the ratio of between-class scatter to within-class scatter in the direction defined by the corresponding eigenvectors after projecting the data. These eigenvectors define the projection directions from the original high-dimensional space to the new low-dimensional space. The purpose of this step is to find a matrix that, when multiplied by the data points, maximizes between-class scatter while minimizing within-class scatter;
• Select eigenvectors with the largest eigenvalues: Choose the eigenvectors corresponding to the largest eigenvalues. These eigenvectors determine the optimal new coordinate axes;
• Project the data onto the new coordinate axes: project the original data onto the new coordinate system using the selected eigenvectors to achieve dimensionality reduction.

To improve recognition accuracy and classification precision in hyperspectral data analysis, more complex neural network structures can be introduced when building models. Deep Convolutional Neural Networks (DCNN) or Recurrent Neural Networks (RNN) can be utilized to enhance the learning ability for complex data patterns. By combining multiple base models such as random forests, gradient boosting trees, etc., and using techniques like voting and weighted averaging, the predictive accuracy can be further enhanced. Leveraging knowledge from the source domain to aid learning in the target domain can improve model performance in cases of scarce data or high labeling costs. Adjusting the learning rate to enhance training speed and performance, such as using AdaGrad, Adam, etc., can help prevent getting trapped in local optima.

4.4.2. Deep Learning Methods

Deep learning is a type of machine learning method based on artificial neural networks that utilizes multiple layers of neural networks to perform nonlinear transformations and feature extraction, enabling the modeling and processing of complex data [106]. Common deep learning algorithms used for hyperspectral data analysis include convolutional neural networks, deep neural networks, and residual networks, among others [107,108].

• Convolutional Neural Networks

Convolutional Neural Networks (CNNs) are a type of feedforward neural network that include convolutional computations and are particularly well-suited for tasks such as image processing [109]. CNNs have the capability of learning representations and can perform translation-invariant classification of input information based on their hierarchical structure, hence also known as “translation-invariant artificial neural networks”.

The working principle involves first extracting various features from the input using convolutional and pooling layers, and then combining these features using fully connected layers to obtain the final output [110]. The convolution operation involves moving a small window (data window) over the image, multiplying and summing elements element-wise. This window essentially consists of a set of fixed weights and can be viewed as a specific filter or convolutional kernel.

The convolutional layer is the core component of convolutional neural networks, enabling feature extraction from input images through convolutional operations. In the convolution operation, each neuron in the convolutional layer performs convolution operations with a portion of pixels in the input image, resulting in a feature map [111]. Convolution operations help capture local features in the input image, such as edges, textures, and other image characteristics.

The pooling layer is another essential part of convolutional neural networks, reducing the complexity of the model by downsampling the feature maps. Pooling operations typically include max pooling and average pooling, where max pooling involves selecting the maximum value within a specific region, while average pooling involves computing the average value within a specific region. Pooling operations help reduce the number of parameters in the model, thereby enhancing the model’s generalization ability.

• Deep Neural Networks

Deep Neural Networks (DNNs) are recursive functions composed of multiple layers, each consisting of multiple neurons. Each neuron receives outputs from all neurons in the previous layer, calculates outputs based on input data, and passes them to neurons in the next layer, ultimately completing prediction or classification tasks [112,113]. The basic steps of deep neural networks are as follows:

• Data preprocessing: Transforming raw data into a format acceptable by the model. Common preprocessing methods include normalization, standardization, encoding, etc.;
• Building network structure: Deep neural networks consist of multiple layers, including input layers, hidden layers, and output layers. The input layer receives input data, hidden layers can contain multiple levels based on the model’s complexity, and the output layer is responsible for outputting the model’s prediction results;
• Defining the loss function: The loss function measures the gap between the model’s prediction results and the ground truth. Common loss functions include Mean Squared Error (MSE), Cross Entropy, etc.;
• Backpropagation: through the backpropagation algorithm, the model’s parameters are updated layer by layer based on the gradient information of the loss function to minimize the loss function;
• Model training: input training data into the model, continuously update parameters through the backpropagation algorithm until the model converges or reaches a predefined stopping condition;
• Model evaluation: evaluate the model’s performance using an independent test dataset, including metrics such as accuracy, precision, recall, etc.

• Residual Networks

Residual Networks (ResNets) make deep neural networks easier to optimize and improve accuracy by introducing residual blocks and skip connections. In traditional neural networks, the output of each layer comes from the output of the previous layer [114]. However, in residual networks, the output of each layer is obtained by adding the output of the previous layer to the input of that layer. This residual connection can be viewed as a skip connection that directly passes information from one layer to the next. Each residual block consists of several convolutional layers and a skip connection that adds the input directly to the output of the convolutional layers, forming a residual connection [115]. This design aims to enable the network to learn residual functions, the differences between inputs and outputs, rather than directly learning the mapping from inputs to outputs. This residual learning concept helps alleviate the vanishing gradient problem and makes the network easier to optimize.

The basic unit of residual networks is the residual module, which includes two or more convolutional layers and a skip connection [116]. The skip connection adds the input directly to the output of the convolutional layers, forming a residual connection. An example of a residual block is shown in Figure 9.

5. Specific Applications and Analyses

In the case studies mentioned above, the combination of hyperspectral technology with convolutional neural networks has achieved significant results in agricultural applications. In the data preprocessing process, the S–G smoothing filter algorithm and second-order derivative method were used for spectral preprocessing. For data analysis, various classification models including CNN, LS-SVM, RF, KNN, and LDA were established with spectral data as input variables and disease severity levels as output variables. By integrating hyperspectral imaging technology with CNN, the successful identification and classification of potato late blight incubation period samples were achieved. The classification models of CNN, LS-SVM, RF, KNN, and LDA were compared, with overall accuracy rates of 99.68%, 90.77%, 92.30%, 93.10%, and 92.34%, respectively. The CNN model performed the best with accuracy, sensitivity, and specificity reaching 99.76%, 98.82%, and 99.54%, respectively, achieving a high identification rate of 99.73% for incubation period samples, which was 5.55%–14.15% higher than other methods [117].

In another case study by Zhao Sen et al., hyperspectral imaging technology was used for the early detection and identification of black spot disease in Acanthopanax. Data preprocessing involved removing brightness and darkness noise, applying smoothing, and using principal component analysis for data dimensionality reduction. Different kernel functions of support vector machine (SVM) were employed for classification, with the polynomial kernel function achieving the best classification accuracy of 92.77% [118].

Furthermore, in the study by Huang Yiqi et al., hyperspectral imaging technology was used for the early detection and identification of rust disease and leaf spot disease in sugarcane. Preprocessing of the spectral data involved first-order derivative, convolution smoothing, and standard normal transformation. Various SVM and random forest (RF) models were established based on different preprocessing and dimensionality reduction methods. The SVM model with SG-SVM preprocessing achieved the best classification accuracy of 99.65%, while the RF model with SG-RF preprocessing had an accuracy of 97.94% [119].

In conclusion, the accuracy of identification and classification models in hyperspectral technology applications is significantly influenced by the preprocessing steps. Effective preprocessing of hyperspectral data can enhance model accuracy, and selecting appropriate preprocessing methods can improve the accuracy of identification models. When establishing identification and classification models, choosing between machine learning and deep learning methods based on data characteristics can boost accuracy. Each method has its advantages and disadvantages, with machine learning being simple and flexible but limited in handling complex tasks, while deep learning excels in learning complex patterns but requires more data and computational resources. Flexibility in choosing preprocessing and data analysis methods is crucial for the effective application of hyperspectral imaging technology in agricultural tasks.

6. Concluding Remarks

Hyperspectral imaging technology plays an important role in the field of agriculture, and can be used for crop growth monitoring, pest and disease diagnosis, soil nutrient analysis, agricultural product quality testing, and crop variety identification. Through hyperspectral image analysis, it is possible to accurately assess the health status of crops, diagnose pest and disease issues, estimate soil nutrient content, analyze agricultural product quality indicators, and identify crop varieties, providing comprehensive data support for agricultural production. However, there are some challenges in the application of hyperspectral imaging technology in agriculture, including high equipment costs, complex data processing requirements, the impact of environmental factors on results, and a lack of standardization. Existing research also has some shortcomings: (1) insufficient sample data leading to inadequate model accuracy and generalization capabilities; (2) high-dimensional hyperspectral data making feature extraction and selection difficult, requiring further research on effective methods; (3) some technologies overly rely on specific datasets and conditions, leading to limited model generalization capabilities; and (4) inadequate real-time performance and operability, necessitating faster and more convenient data acquisition and processing methods [120,121,122].

With the rapid development of hyperspectral imaging technology and artificial intelligence, their integration will expand into a wide range of application areas. As technology innovation and hardware performance improve, hyperspectral data collection will become more convenient and efficient. The use of drones equipped with hyperspectral cameras for data collection has been widely adopted in various fields. Many researchers have focused on preprocessing and analyzing hyperspectral data, improving and designing more algorithms to enhance data processing capabilities and analysis effectiveness.

Agriculture holds a significant position globally, and the transition from traditional agriculture to smart agriculture reflects its rapid development. The rise of smart agriculture, combined with advanced sensing technology, communication, artificial intelligence, and other technologies, is propelling agriculture into a high-speed development era. Hyperspectral imaging technology is increasingly applied in agriculture, particularly in crop pest and disease identification, and product classification and identification. Combining artificial intelligence, communication, and networking technologies allows for the simultaneous detection of multiple parameters of crops and products, enabling real-time dynamic monitoring. In terms of agricultural product quality classification and identification, simultaneous internal and external quality testing will improve classification accuracy.

In the future, technology improvements and cost reductions will also be the direction of development for hyperspectral imaging technology. As technology continues to advance, the cost of hyperspectral imaging equipment is expected to gradually decrease, while the performance of the equipment will continue to improve. Improvements such as increasing spectral resolution and imaging speed, will make it easier to popularize and apply this technology in agricultural production. The integration of technologies such as the internet of things, big data, and artificial intelligence will also be the direction of development for hyperspectral imaging technology in agricultural production [123].