To address the challenges in estimating forage P content outlined above, two primary issues require attention: (1) the limitations of multispectral data mining methods, which arise from their discrete spectral sampling approach, fail to capture the inherent continuous geometric relationships within multispectral data, thereby limiting the accurate extraction of continuous spectral features; and (2) traditional symbolic regression algorithms based on genetic programming exhibit shortcomings such as dimensionality issues, low search efficiency, and limited global search capability. This section introduces two methods (
Figure 2(2)) to tackle these challenges. First, the CHSF extraction method, developed using “Graph Theory”, enables the generation of continuous spectral curves directly from multispectral data. It creates CHSF images (CHSFIs) with multiple spectral dispersions using various sampling approaches. Secondly, integrating deep reinforcement learning and genetic programming into symbolic regression enhances global search capabilities and efficiency (
Figure 2(3)).
Figure 2.
Framework of this study. (1) We collected data including DEM, multispectral images, and ground sampling. (2) We defined the CHSF and generated a method base and CHSFI dataset. (3) Using the DRL-GP algorithm, we built a symbolic regression model to acquire the optimal symbolic inversion model and spectral dispersion. (4) We analyzed the spatiotemporal dynamics of P content during growing seasons.
2.3.1. Extracting CHSF from Sentinel-2 MSI of Forage P Content Estimation
Sunlight reflection on surfaces can generally be visualized as a continuous curve. However, due to the discrete sampling methods of multispectral sensors, current data mining approaches remain limited to discrete points [
35]. To address this limitation, this study leverages “Graph Theory” [
36], which has been applied in various fields, including algebraic topology [
37], quantum computing [
38], probability statistics [
39], and transportation planning [
40]. “Graph Theory” has also been increasingly applied in the field of remote sensing. For example, Xie et al. used remote sensing data and “Graph Theory” to identify urban structures [
41]; Wang et al. extracted river width from remote sensing images based on “Graph Theory” [
42]; and Matthias et al. applied “Graph Theory” to extract agricultural fields from remote sensing imagery [
43]. Building on these applications, we leverage “Graph Theory” for feature mining in remote sensing imagery.
“Graph Theory” is commonly employed to model relationships between events, where vertices represent individual events, and edges connect vertices to indicate relationships [
44] (
Figure 2(2)—“Graph Theory”). From this perspective, the spectral curve of each pixel in multispectral or hyperspectral remote sensing images can be represented as a network of interconnected components. Shifting from the original one-dimensional “sequence” to a two-dimensional “image”, pixel values from different spectral bands in multispectral images are treated as “points,” and the edges between them represent the spectral variation in different land features. As a result, the spectral sequences form a “polygonal graph”. However, the multispectral/hyperspectral graph exhibits discrete connectivity due to the sensor’s discrete spectral sampling (
Figure 2(2)—remote sensing imagery graph). To overcome this, we compute continuous spectral curves, forming a “curvilinear graph” (
Figure 2(2)—remote sensing imagery graph). This method, known as CHSF extraction, allows for the generation of continuous spectral curves from multispectral data (
Figure 2(2)—remote sensing imagery graph). Furthermore, we produce 10 CHSFIs with varying spectral dispersions using equidistant discrete sampling (1–10 nm) (
Figure 2(2)—CHSFI dataset) to estimate the content of forage P.
In the context of “Spectral Graph”, numerous ways exist to connect different points graphically. However, not all curves can be considered CHSF curves for remote sensing imagery. The extraction of these curves must align with the fundamental principles of remote sensing image acquisition and the characteristics of terrestrial features [
45]. The proposed CHSF aims to establish an unsupervised system for extracting continuous hyperspectral features. Assuming a set of n known points (representing the number of spectral bands in the spectral image) denoted as
, the CHSF extraction involves computing the corresponding function values
for
, where (
).
is a function defined on the interval [a, b] (where [a, b] represents the spectral range of input imagery),
are n distinct points on [a, b], and let G be a given class of functions. If there exists function
in G satisfying the following equation, then
is considered the CHSF methods of
with respect to the points
:
where
refers to the number of bands,
is a different method of CHSFs, and
is the position of each band.
The CHSF, depicted in
Figure 2(2)—(CHSF method base), categorizes points into two types: those lying on the curve and those off the curve. This section focuses on the extraction of spectral features using two primary methodological approaches: interpolation and fitting methods. Interpolation methods encompass spline interpolation (quadratic, cubic, and quartic), radial basis function (RBF) interpolation (multiple quadratic, inverse polynomial and thin plate), and polynomial function interpolation (linear and Hermite). Conversely, fitting methods consist of polynomial fitting (quadratic, and cubic), piecewise polynomial fitting (linear, quadratic, and cubic), and machine learning-based fitting (k-nearest neighbors, decision tree, and random forest). Interpolation refers to the process of estimating new data points within a given range based on known discrete points [
46], addressing issues related to points lying on a continuous curve. Fitting, in contrast, involves approximating the original point function curve within a range using known discrete points, which helps resolve the challenge of points not lying on the curve [
47]. This section expands the sixteen methods to spectral data and Sentinel-2 multispectral imagery (MSI) to construct CHSF datasets.
Initially, grassland spectral data from the USGS were utilized to establish the CHSF method base. This base was instrumental in extracting CHSFIs with a spectral dispersion of 5 nm from multispectral data obtained through satellite remote sensing imagery.
Figure 3 illustrates the challenge of “points lying on the line” and demonstrates the application of interpolation methods within the CHSF framework. The key observations are as follows: (1) Spline Interpolation: Quadratic, cubic, and quartic splines were utilized. Beyond quartic splines, the CHSF curves exhibited negative trends, deviating from the expected reflection mechanisms in grassland remote sensing. Quadratic and cubic splines shared similar characteristics, displaying multiple wave crests and troughs along their profiles. (2) RBF Interpolation: Compared to spline interpolation, RBF methods showed more pronounced wave crests and troughs across the spectral range. Multiple quadratic and inverse polynomial basis function interpolations exhibited distinct features between 470 nm and 1045 nm, while the thin-plate basis function interpolation revealed specific peaks and troughs at 1621.5 nm and 1923.6 nm. (3) Polynomial Interpolation: Linear and Hermite interpolations produced smoother curves compared to spline and RBF methods. Although smoother, these profiles still adhered to the fundamental principles of grassland remote sensing reflection.
Figure 4 visually represents the “points lying off the line” problem through the application of various fitting methods: (1) Machine learning fitting: CHSF curves generated using a decision tree and K-nearest neighbor algorithms tend to be relatively simple, while random forest produces more varied results. In general, machine learning fitting lacks the distinct wave crests and troughs typically observed in spectral curves. Instead, it tends to display a noticeable step-like pattern. (2) Piecewise polynomial fitting: When the polynomial degree exceeds three, the resulting CHSF curve deviates from the expected grassland remote sensing reflection mechanisms, often falling below zero. Therefore, piecewise linear, quadratic, and cubic polynomials are applied. The CHSF curve generated by linear polynomial fitting lacks evident wave crests and troughs. In contrast, quadratic and cubic piecewise polynomial fitting produce curves with clear wave crests and troughs, concentrated in the 560 nm to 880 nm range (
Figure 4). (3) Polynomial fitting: Similarly, when the polynomial degree is greater than three, the CHSF curve falls below 0, and linear fitting results in a straight line. Therefore, quadratic and cubic polynomials are selected. While these methods generate CHSF curves with noticeable wave crests and troughs, they tend to diverge from the fundamental trends in grassland spectral reflection.
In summary, these CHSF curves are derived from equidistant spectral discrete sampling, generating a dataset of CHSFIs with varying spectral dispersions (ranging from 1 to 10 nm) based on Sentinel-2 MSI data. The detailed process for this CHSF extraction and dataset generation is illustrated in
Figure 5.
2.3.2. Symbolic Regression Based on DRL-GP of Forage P Content Estimation
Genetic algorithms have been widely employed in symbolic regression problems because of their powerful search capabilities. However, they often struggle to efficiently capture relevant knowledge during the evolutionary process, requiring significant computational resources and time for each iteration. To fully exploit the advantages of genetic algorithms in symbolic regression, this section integrates deep learning and reinforcement learning algorithms. Reinforcement learning is a machine learning algorithm that employs rewards and penalties to enable an agent to learn through interactions with its environment. Its goal is to discover the optimal strategy and maximize cumulative rewards [
48]. In this study, deep reinforcement learning is embedded within the genetic algorithm framework, assigning an agent to each gene point of symbolic expressions. The specific process consists of three key steps (
Figure 2(3)): state input, action space definition, and searching for the optimal expression.
The neural network uses all training data as input. The input specifically includes CHSFs for each measured sample plot, and the output corresponds to the measured P content for each plot (
Figure 5). To improve the efficiency of the entire learning process, all images are converted into sequential inputs.
The action space for the skeletal component of symbolic expressions consists of a predefined function library. As for the feature component of symbolic expressions, the action space encompasses all input parameters. These two components are mutually exclusive, and the skeletal and feature action spaces do not intersect.
In each generation, a symbolic expression is generated, and this experiment utilizes R
2, MSE, and MAE as reward metrics for the reinforcement learning algorithm. Iterations continue until the highest coefficient of determination (R
2) (Equation (2)), lowest mean squared error (MSE) (Equation (3)), and mean absolute error (MAE) (Equation (4)) values are determined, serving as criteria for identifying the optimal symbolic expression.
where
refers to the sum of squared errors between individual observed values
and their corresponding predicted values
and
are the sum of squares between each observed value and their mean.
For training, ten-fold cross-validation is utilized to mitigate the influence of data overfitting or underfitting on model performance. The hyperparameters are configured as follows: PyTorch 1.10.2 framework is employed, and the loss function is defined as a combination of the coefficient of determination R2, MSE, and MAE.
In this experiment, we selected a training size of 75% and a test size of 25%. During the selection phase, we used a tournament selector with a size of five participants. The population size was set to 100 because the selection operator chose individuals from both the population and the archive. The evaluation times were set to 10,000 and the number of evolutionary productions to 100, contingent on the population size. This study employed a neural network with two fully connected layers, comprising a total of 128 neurons.
For the skeleton of the symbolic expressions, the function space includes basic mathematical operators such as addition (+), subtraction (−), multiplication (×), division (÷), and trigonometric functions (sin, cos, and tan).