Monitoring Forest Health Using Hyperspectral Imagery: Does Feature Selection Improve the Performance of Machine-Learning Techniques?

Abstract

This study analyzed highly correlated, feature-rich datasets from hyperspectral remote sensing data using multiple statistical and machine-learning methods. The effect of filter-based feature selection methods on predictive performance was compared. In addition, the effect of multiple expert-based and data-driven feature sets, derived from the reflectance data, was investigated. Defoliation of trees (%), derived from in situ measurements from fall 2016, was modeled as a function of reflectance. Variable importance was assessed using permutation-based feature importance. Overall, the support vector machine (SVM) outperformed other algorithms, such as random forest (RF), extreme gradient boosting (XGBoost), and lasso (L1) and ridge (L2) regressions by at least three percentage points. The combination of certain feature sets showed small increases in predictive performance, while no substantial differences between individual feature sets were observed. For some combinations of learners and feature sets, filter methods achieved better predictive performances than using no feature selection. Ensemble filters did not have a substantial impact on performance. The most important features were located around the red edge. Additional features in the near-infrared region (800–1000 nm) were also essential to achieve the overall best performances. Filter methods have the potential to be helpful in high-dimensional situations and are able to improve the interpretation of feature effects in fitted models, which is an essential constraint in environmental modeling studies. Nevertheless, more training data and replication in similar benchmarking studies are needed to be able to generalize the results.

1. Introduction

The use of machine learning (ML) algorithms for analyzing remote sensing data has seen a huge increase in the last decade [1]. This coincided with the increased availability of remote sensing imagery, especially since the launch of the first Sentinel satellite in the year 2014. At the same time, the implementation and usability of learning algorithms has been greatly simplified with many contributions from the open-source community. Scientists can nowadays process large amounts of (environmental) information with relative ease using various learning algorithms. This makes it possible to easily extend benchmark comparison matrices of studies in a semi-automated way, possibly stumbling upon unexpected findings, such as process settings, that would not have been explored otherwise [2].

ML methods in combination with remote sensing data are used in many environmental fields, such as vegetation cover analysis and forest carbon storage mapping [3,4]. The ability to predict these environmental variables in unknown regions qualifies ML algorithms as a helpful tool for such environmental analyses. One aspect of this research field is to enhance the understanding of biotic and abiotic stress triggers, for example, by analyzing tree defoliation [5]. Defoliation is known to be a proxy for pathogen and insect damage [6]. While common symptoms are observable across species, some effects and their degree of severity are species-specific [7]. Defoliation has also been shown to increase predisposition of tree death from secondary biotic factors up to ten years after the actual defoliation event [8]. Other approaches for analyzing forest health include change detection [9] or describing the current health status of forests on a stand level [10].

Vegetation indices have shown the potential to provide valuable information when analyzing forest health [11,12]. Most vegetation indices were developed with the aim of being sensitive to changes in specific wavelength regions, serving proxies for underlying plant physiological processes. In some cases, indices developed for different purposes than the one to be analyzed (e.g., defoliation of pine trees) may help to explain complex underlying relationships that are not obvious at first. This emphasizes the need to extract as much information as possible from the available input data to generate promising features that can help in improving our understanding of the modeled relationship [13]. A less known index type that can be derived from spectral information is the normalized ratio index (NRI). In contrast to most vegetation indices, NRIs do not use an expert-based formula following environmental heuristics; instead, they make use of a data-driven feature engineering approach by combining (all possible) combinations of spectral bands [14]. When working with hyperspectral data, thousands of NRI features can be derived this way.

Even though ML algorithms are capable of handling highly correlated input variables, model fitting becomes computationally more demanding and model interpretation more challenging. Feature selection approaches can help to address this issue, reducing possible noise in the feature space, simplifying model interpretability, and possibly enhancing predictive performance [15]. Instead of using wrapper feature selection methods, which add a substantial overhead to a nested model optimization approach, especially for datasets with many features, this study focuses on (ensemble) filter methods, which can be directly integrated into the hyperparameter optimization step during model construction.

Overall, this study aims to show how high-dimensional datasets can be handled effectively with ML methods while still allowing the interpretation of the fitted models. The predictive power of non-linear methods and their ability to handle highly correlated predictors is combined with common and new approaches for assessing feature importance and feature effects. However, this study focuses mainly on investigating the effects of filter methods and feature set types on predictive performance rather than interpreting individual feature effects.

Considering these opportunities and challenges, the research questions of this study are as follows:

• How do different feature selection methods influence the predictive performance of ML models of the defoliation of trees?
• Do different (environmental) feature sets show differences in performance?
• Can predictive performance be substantially improved by combining feature sets?
• Which features are most important and how can these be interpreted in this context?

In recent years, various studies that have used hyperspectral data to analyze pest/fungi infections on trees have been published. Pinto et al. [16] successfully used hyperspectral imagery to characterize pest infections on peanut leaves using random forest, while Yu et al. [17] aimed to detect pine wilt disease in pine plots in China using vegetation indices derived from hyperspectral data. Other studies which applied hyperspectral data for forest health monitoring are [18,19,20]. Building upon these successful applications of hyperspectral remote sensing usage in the field of leaf and tree health monitoring, this work analyzes tree defoliation in northern Spain using airborne hyperspectral data. The methodology of this study uses ML methods in combination with feature selection and hyperparameter tuning. In addition, feature importance was analyzed. Incorporating the idea of creating data-driven NRIs, this study also discusses the practical problems of high dimensionality in environmental modeling [21,22].

2. Materials and Methods

2.1. Data and Study Area

Airborne hyperspectral data with a spatial resolution of one meter and 126 spectral bands were available for four Monterey Pine (Pinus radiata D. Don) plantations in northern Spain. The trees in the plots suffer from infections with pathogens such as Diplodia sapinea (Fr.) Fuckel, Fusarium circinatum Nirenberg & O’Donnell, Armillaria mellea (Vahl) P. Kumm, Heterobasidion annosum (Fr.) Bref, Lecanosticta acicola (Thüm) Syd., and Dothistroma septosporum (Dorogin) M. Morelet causing (among others) needle blight, pitch canker, and root diseases [23,24]. The first two fungi are mainly responsible for the foliation loss of the trees analyzed in this study [25]. In situ measurements of defoliation of trees (serving as a proxy for tree health) were collected by visual inspection by experts. Defoliation in percent was used as the response variable (Figure 1).

It is assumed that the fungi infect the trees through open wounds, possibly caused by previous hail damage [25]. The dieback of these trees, which are mainly used as timber, causes substantial economic losses [26].

2.1.1. In Situ Data

The Pinus radiata plots of this study, namely Laukiz1, Laukiz2, Luiando, and Oiartzun, are located in the northern part of the Basque Country (Figure 2). Oiartzun has the largest number of observations ($n=559$ trees), while Laukiz2 is the largest in area (1.44 ha). All plots besides Luiando are located within 100 km from the coast (Figure 2). A total of 1808 observations are available (Laukiz1: 559, Laukiz2: 451, Luiando: 301, Oiartzun: 497). Field surveys were conducted in September 2016 by experienced forest pathologists. Defoliation was measured in 5% steps through visual inspection with the help of a score card. For Laukiz2, values at three height levels (bottom, mid, and top) were available and averaged into an overall defoliation value, resulting in values that are not multiples of 5% (e.g., 8.33%). The magnitude of human observer errors in such surveys, including the present one, is not precisely known and has being discussed for many years [27]. Ref. [28] estimated human observer errors in defoliation surveys to range between 7% and 18%.

2.1.2. Hyperspectral Data

The airborne hyperspectral data were acquired by an AISA EAGLE-II sensor during two flight campaigns on 28 September and 5 October 2016 at noon. All preprocessing steps (geometric, radiometric, and atmospheric) were conducted by the Institut Cartografic i Geologic de Catalunya (ICGC). The first four bands were corrupted, leaving 122 bands with valid information. Additional metadata information is available in

Table 1. Specifications of hyperspectral data.

Characteristic	Value
Geometric resolution	1 m
Radiometric resolution	12 bit
Spectral resolution	126 bands (404.08–996.31 nm)
Correction:	Radiometric, geometric, atmospheric

.

2.2. Derivation of Indices

To use the full potential of the hyperspectral data, all possible vegetation indices supported by the R package hsdar (89 in total) as well as all possible NRI combinations were calculated. NRIs follow the optimized multiple narrow-band reflectance (OMNBR) concept of data-driven information extraction from narrow-band indices of hyperspectral data [13,14]. While various index formulations, such as band ratios or normalized ratios, are available, indices involving the same bands are strongly correlated. Since the widely used NDVI index belongs to the group of normalized ratio indices (NRIs), which are implemented in the hsdar R package, we used the following normalized difference index (NDI) formula to combine all pairs of reflectances:

$$NR{I}_{i,j}=\frac{ban{d}_i-ban{d}_j}{ban{d}_i+ban{d}_j}$$

(1)

where i and j are the respective band numbers.

To account for geometric offsets within the hyperspectral data, which were reported by ICGC to be potentially up to one meter, a buffer of one meter radius around the centroid of each tree was used when extracting the reflectance values. A pixel was considered to fall into a tree’s buffer zone if the centroid of the respective pixel was touched by the buffer. The pixel values within a buffer zone were averaged and formed the final reflectance value of a single tree, and they were used as the base information to derive all additional feature sets. In total, $\frac{121\ast 122}{2}=7381$ NRIs were calculated.

2.3. Feature Selection

High-dimensional, feature-rich datasets come with several challenges for both model fitting and evaluation:

• Model fitting times increase.
• Noise is possibly introduced into models by highly correlated variables [29].
• Model interpretation and prediction become more challenging [29].

To reduce the feature space of a dataset, several conceptually differing approaches exist: wrapper methods, filters, penalization methods (lasso and ridge), and principal component analysis (PCA) [30,31,32,33]. In contrast to wrapper methods, filters have a much lower computational cost, and their tuning can be added to the hyperparameter optimization step. In addition, filters are less known than wrapper methods, and, in recent years, ensemble filters, which have shown promising results compared to single filter algorithms, were introduced [34]. These two points mainly led to the decision to focus on filter methods for this work and evaluate their effectiveness on highly correlated, high-dimensional datasets. Due to this focus, only this subgroup of feature selection methods is be introduced in greater detail in the following sections.

2.3.1. Filter Methods

The concept of filters originates from the idea of ranking features following a score calculated by an algorithm [32]. Some filter methods can only deal with specific types of variables (e.g., numeric or nominal). Filters only rank features; they do not decide which covariates to drop or keep [35]. The decision of which features to keep for model fitting can be integrated into the optimization phase during model fitting, along with hyperparameter tuning. Thus, the number of top-ranked features to be included in the model is treated as an additional hyperparameter of the model. This hyperparameter is tuned to optimize the model’s performance.

Beyond the use of individual filter methods to rank and select features, recent studies have shown that combining several filters by using statistical operations such as “minimum” or “mean” may enhance the predictive performance of the resulting models, especially when applied to multiple datasets [34,36]. This approach is referred to as “ensemble filtering” [37]. Ensemble filters align with the recent rise of the ensemble approach in ML, which uses the idea of stacking to combine the predictions of multiple models, aiming to enhance predictive performance [38,39,40]. In this work, the Borda ensemble filter was used [34]. Its overall feature order is determined by the sum of filter ranks of all individual filters in the ensemble.

Filter methods can be categorized based on three binary criteria: multivariate or univariate feature use, correlation or entropy-based importance weighting, and linear and non-linear filter methodology. Care needs to be taken to not weigh certain classes more than others in the ensemble, as, otherwise, the ranking will be biased. In this study, this was taken care of by checking the rank correlations (Spearman’s correlation) of the generated feature rankings of all methods against each other. If filter pairs showed a correlation of 0.9 or higher, only one of the two was included in the ensemble filter, selecting it at random.

2.3.2. Description of Used Filter Methods

Filter methods can be classified as follows (

Table 2. List of filter methods used in this work, their categorization, and scientific reference.

Name	Group	Ref.
Linear correlation (Pearson)	univariate, linear, correlation	[41]
Information gain	univariate, non-linear, entropy	[42]
Minimum redundancy, maximum relevance	multivariate, non-linear, entropy	[43]
Carscore	multivariate, linear, correlation	[44]
Relief	multivariate, linear, entropy	[45]
Conditional minimal information maximization	multivariate, linear, entropy	[46]

):

• Univariate/multivariate (scoring based on a single variable/multiple variables).
• Linear/non-linear (usage of linear/non-linear calculations).
• Entropy/correlation (scoring based on derivations of entropy or correlation-based approaches).

The filter “Information Gain” in its original form is only defined for nominal response variables:

$$H\left(Class\right)+H\left(Attribute\right)-H(Class,Attribute)$$

(2)

where H is the conditional entropy of the response variable (class or Y) and the feature (attribute or X). $H\left(X\right)$ is Shannon’s entropy [47] for a variable X, and $H(X,Y)$ is a joint Shannon’s entropy for a variable X with a condition to Y. $H\left(X\right)$ itself is defined as

$$H\left(X\right)=-\sum _{i=1}^{n}P\left(x_i\right){log}_{b}P\left(x_i\right)$$

(3)

where b is the base of the logarithm used, most commonly 2.

In order to use this method with a numeric response (percentage defoliation of trees), the variable was discretized into equal bins $n_{bin}$ = 10 and treated as a categorical variable.

2.4. Benchmarking Design

2.4.1. Algorithms

The following learners were used in this work:

• Extreme gradient boosting (XGBoost);
• Random forest (RF);
• Penalized regression (with L1/lasso and L2/ridge penalties);
• Support vector machine (SVM, radial basis function Kernel);
• Featureless learner.

RF and SVM are well-established algorithms and widely used in environmental remote sensing. Extreme gradient boosting (commonly abbreviated as XGBoost) has shown promising results in benchmarking studies in recent years. Penalized regression is a statistical modeling technique capable of dealing with highly correlated covariates by penalizing the model coefficients [48]. Common penalties are “lasso” (L1) and “ridge” (L2). Ridge regression does not remove variables from the model (penalization to zero), but it shrinks them towards zero, keeping them in the model. A featureless learner was included for a baseline comparison.

In total, the benchmarking grid consisted of 156 experiments (6 feature sets × 3 ML algorithms × 8 feature-selection methods and for the L1/L2 models, 6 feature sets × 2 models. The selected hyperparameter settings are shown in Appendix A

Table A1. Hyperparameter ranges and types for each model. Hyperparameter notations from the respective R packages are shown.

Model (Package)	Hyperparameter	Type	Start	End	Default
RF (ranger)	$x_{try}$	dbl	0	0.5	-
	min.node.size	int	1	10	1
	sample.fraction	dbl	0.2	0.9	1
SVM (kernlab)	C	dbl	$2^{-10}$	$2^{10}$	1
	$\sigma$	dbl	$2^{-5}$	$2^5$	1
XGBoost (xgboost)	nrounds	int	10	70	-
	colsample_bytree	dbl	0.6	1	1
	subsample	dbl	0.6	1	1
	max_depth	int	3	15	6
	gamma	int	0.05	10	0
	eta	dbl	0.1	1	0.3
	min_child_weight	int	1	7	1

. All code and data are included in the research compendium of this study (https://doi.org/10.5281/zenodo.2635403 (accessed on 22 November 2021).

2.4.2. Feature Sets

Three feature sets were used in this study, each representing a different approach to feature engineering:

• The raw hyperspectral band information (HR): no feature engineering;
• Vegetation indices (vegetation index (VI)s): expert-based feature engineering;
• Normalized ratio indices (NRIs): data-driven feature engineering.

The idea of splitting the features into different sets originated from the question of whether feature-engineered indices derived from reflectance values have a positive effect on model performance. Peña et al. 2017 [49] is an exemplary study that used this approach in a spectro-temporal setting. Benchmarking learners on these feature sets while keeping all other variables, such as model type, tuning strategy, and a partitioning method, fixed makes it possible to draw conclusions on their individual impact. Each feature set has distinct capabilities that differentiate it from the others. This can have both a positive and negative effect on the resulting performance, which is one question this study aims to explore. For example, feature set VI misses certain parts of the spectral range, as the chosen indices only use specific spectral bands. Feature set NRI will introduce highly correlated features, for which some algorithms may be more suitable than others.

In addition to these individual feature sets, the following combinations of feature sets were also compared:

• HR + VI
• HR + NRI;
• HR + VI + NRI.

Some individual features were removed before using the datasets for modeling when being numerically equivalent to another feature based on the pairwise correlation being greater than $1-{10}^{-10}$. This preprocessing step reduced the number of covariates for feature set VI to 86 (from 89). This decision was made to prevent numerical issues that may occur in the subsequent tuning, filtering, and model fitting steps when offering features with a pairwise correlation of (almost) one. The remaining features were then used as input for the filter-based feature selection within the CV.

2.4.3. Hyperparameter Optimization

Hyperparameters were tuned using model-based optimization (MBO) within a nested spatial cross-validation (CV) [50,51,52]. In MBO, first, n hyperparameter settings are randomly chosen from a user-defined search space. After these n settings have been evaluated, one new setting, which is evaluated next, is proposed by a fitted surrogate model (by default, a kriging method). This strategy continues until a user-defined stopping criterion is satisfied [53,54].

In this work, an initial design of 30 randomly drawn hyperparameter settings in combination with a stopping criterion of 70 iterations was used, resulting in a total budget of 100 evaluated hyperparameter settings per fold. The advantage of this tuning approach is a substantial reduction of the tuning budget that is required to find a setting close to the global optimization minimum. MBO may outperform methods that do not use information from previous iterations, such as random search or grid search [55]. Tuning ranges used in this work are shown in Table A1.

To optimize the number of features used for model fitting, the percentage of features was added as a hyperparameter during the optimization stage [51]. For PCA, the number of principal components was tuned. The RF hyperparameter $m_{try}$ was re-expressed as $m_{try}=p_{\mathrm{sel}}^t$, a function of the number of selected features, $p_{\mathrm{sel}}$. It was thus tuned on a logarithmic scale by varying t between 0 (i.e., $m_{try}=1$) and 0.5 (i.e., $m_{try}=\sqrt{p_{\mathrm{sel}}}$). This was necessary to ensure that $m_{try}$ did not exceed the number of features available after optimizing the feature percentage during tuning.

2.4.4. Spatial Resampling

A spatial nested cross-validation on the plot level was chosen to account for spatial autocorrelation within the plots and assess model transferability to different plots [52,56]. The root mean square error (RMSE) was chosen as the error measure. Each plot served as one cross-validation fold, resulting in four iterations in total. The inner level of cross-validation for hyperparameter tuning also used plot-level cross-validation.

2.5. Feature Importance and Feature Effects

Estimating feature importance for datasets with highly correlated features is a challenging task for which numerous model-specific and model-agnostic approaches exist [48,57,58]. The strong correlations among features make it challenging to calculate an unbiased estimate for single features [59]. Methods such as partial dependence plots (PDP) or permutation-based approaches may produce unreliable estimates in such scenarios because unrealistic combinations of feature values are created [59]. The development of robust methods that enable an unbiased estimation of feature importance for highly correlated variables is subject to current research [60].

In this work, permutation-based feature importance was calculated to estimate feature importance or effects [61]. With the limitations in mind when applied to correlated features, the aim was to get a general overview of the feature importance of the hyperspectral bands while trying to avoid an over-interpretation of results. The best-performing algorithm on the HR task (i.e., SVM) was used for the feature importance calculation.

To facilitate interpretation, the ten most important indices of the best performing models using feature sets HR and VI were linked to the spectral regions of the hyperspectral data that went into their calculation. The aim was to visualize the most important features along the spectral curve of the plots to better understand which spectral regions were most important for the model.

2.6. Research Compendium

All tasks of this study were conducted using the open-source statistical programming language R [62]. A complete list of all R packages used in this study can be found in the linked repositories mentioned in the next paragraph. Due to space limitations, only selected packages with high impact on this work are explicitly cited.

The algorithm implementations of the following packages were used: xgboost [63] (extreme gradient boosting), kernlab [64] (support vector machine) and glmnet [65] (penalized regression). The filter implementations of the following packages were used: praznik [66] and FSelectorRcpp [67]. Package mlr [68] was used for all modeling related steps, and drake [69] was used for structuring the work and for reproducibility. This study is available as a research compendium on Zenodo (10.5281/zenodo.2635403, (accessed on 22 November 2021). Apart from the availability of code and manuscript sources, a static webpage is available at https://pat-s.github.io/2019-feature-selection (accessed on 22 November 2021) which includes additional side analyses that were carried out during the creation of this study.

3. Results

3.1. Principal Component Analysis of Feature Sets

PCA was used to assess the complexity of the three feature sets. Depending on the feature set, 95% of the variance is explained by two (HR), twelve (VI), and 42 (NRI) principal component (PC)s. HR features are therefore highly redundant, while the applied feature transformations enrich the data set, at least from an exploratory linear perspective.

3.2. Predictive Performance

Overall, the response variable “tree defoliation” could be modeled with an RMSE of 28 percentage points (p.p.) (Figure 3). SVM showed almost no differences in RMSE across feature sets whereas other learners (RF, SVM, XGBoost, lasso and ridge) differed up to five p.p. (Figure 3). SVM showed the best overall performance with a mean difference of around three p.p. to the next best model (XGBoost) (

Table 3. The overall best individual learner performance across any task and filter method for RF, SVM, XGBoost, lasso, and ridge, sorted ascending by RMSE (p.p.) including the respective standard error (SE) of the cross-validation run. For extttregr.featureless, the task is not applicable and was therefore removed.

	Task	Model	Filter	RMSE	SE
1	NRI-VI	SVM	Info Gain	27.915	18.970
2	NRI	RF	Relief	30.842	12.028
3	HR	XGBoost	Info Gain	31.165	15.025
4	NRI	Lasso-MBO	No Filter	31.165	15.025
5	NRI	Ridge-MBO	No Filter	31.165	15.025
6	-	regr.featureless	No Filter	31.165	15.025

). Performance differences between test folds were large: Predicting on Luiando resulted in an RMSE of 9.0 p.p. for learner SVM (without filter) but up to 54.3 p.p. when testing on Laukiz2 (

Table 4. Test fold performances in RMSE (p.p.) for learner SVM on the HR dataset without using a filter, showcasing performance variance on the fold level. For each row, the model was trained on observations from all others plots but the given one and tested on the observations of the given plot.

	RMSE	Test Plot
1	28.12	Laukiz1
2	54.26	Laukiz2
3	9.00	Luiando
4	21.17	Oiartzun

).

The combination of feature sets showed small increases in performance for some learners. XGBoost scored slightly better on the combined datasets HR-NRI, NRI-VI, and HR-NRI-VI compared to their standalone variants (NRI and VI) (Figure 3). However, the best performances for RF and XGBoost were scored on NRI and HR, respectively. RF showed a substantial performance increase when using only NRI compared to all other feature sets, whereas for XGBoost, the worst performances were associated with the VI- and NRI-only feature sets (Figure 3).

SVM combined with the “Information Gain” filter achieved the best overall performance (RMSE of 27.915 p.p.) (

Table 5. Best ten results among all learner–task–filter combinations, sorted in decreasing order of RMSE (p.p.) and their respective standard error (SE).

	Task	Model	Filter	RMSE	SE
1	NRI-VI	SVM	Info Gain	27.915	18.970
2	NRI	SVM	CMIM	28.044	19.101
3	VI	SVM	Relief	28.082	19.140
4	NRI-VI	SVM	Borda	28.102	19.128
5	HR	SVM	CMIM	28.119	19.123
6	HR	SVM	MRMR	28.119	19.123
7	VI	SVM	Info Gain	28.121	19.123
8	NRI	SVM	PCA	28.121	19.123
9	HR-NRI	SVM	PCA	28.121	19.123
10	HR-NRI-VI	SVM	PCA	28.121	19.123

). Regressions with ridge (L2) and lasso (L1) penalties showed their best performances on the NRI feature set (Table 3). Combining feature sets for lasso and ridge did not help to increase performance, and while there was no substantial difference for lasso, the performance of ridge improved by around two percentage points. XGBoost showed very poor performances for some feature sets and fills the last ten places of the ranking (

Table 6. Worst ten results among all learner–task–filter combinations, sorted in decreasing order of RMSE (p.p.) and their respective standard error (SE).

	Task	Model	Filter	RMSE	SE
1	VI	XGBoost	No Filter	45.366	6.672
2	HR	XGBoost	No Filter	44.982	5.378
3	VI	XGBoost	PCA	44.539	8.187
4	HR	XGBoost	PCA	44.032	6.183
5	NRI	XGBoost	PCA	43.433	9.543
6	HR-NRI	XGBoost	PCA	43.220	2.557
7	HR-NRI-VI	XGBoost	PCA	41.076	9.862
8	VI	RF	CMIM	39.980	10.144
9	VI	RF	Info Gain	39.623	10.616
10	NRI	XGBoost	Pearson	39.492	11.548

). Especially when combined with PCA, the algorithm was not able to achieve adequate performances.

The effects of filter methods on performance differed greatly between the algorithms: SVM showed no variation in performance across filters (Figure 4). The use of filters for RF resulted in a substantial increase in performance in all tasks, especially on the HR feature set where all filters showed an improved score compared to using no filter (Figure 4). XGBoost’s performance depended strongly on feature selection. In two out of six tasks (HR, VI), using no filter resulted in the worst performance. XGBoost showed the highest overall differences between filters for a single task—for feature set HR, the range is up to 13 p.p. (“CMIM” vs. “no filter”) (Figure 4).

When comparing the usage of filters against using no filter at all, there were no instances in which a non-filtered model scored a better performance than the best filtered one (Figure 4). For SVM, all filters and “no filter” achieved roughly the same performance on all tasks.

The Borda ensemble filter was not able to score the best performance in any learner/ filter setting (Figure 5). For RF and XGBoost, it most often ranked within the better half among all filters of a specific task.

The number of features selected during model optimization strongly varied across learners and plots. RF selected the least features of all three learners, and with the exception of Oiartzun, only one or two variables were selected. SVM used 210 features or more in all instances and selected between 16% (Laukiz1) and 81% (Oiartzun) of the features (

Table 7. Selected feature portions during tuning for the best performing learner–filter settings (SVM Relief, RF Relief, XGBoost CMIM) across folds for task HR-NRI-VI, sorted by plot name. “Features (#)” denotes the absolute number of features selected, and “Features (%)” refers to the percentage relative to the overall features available in the training sets for each plot (Laukiz1 = 1249, Laukiz2 = 1357, Luiando = 1507, Oiartzun = 1311). Results were estimated in a separate model tuning step, not within the main cross-validation comparison.

Learner	Test Plot	Features (%)	Features (#)
RF Car	Laukiz1	0.00245	1/1249
	Laukiz2	0.00359	1/1357
	Luiando	0.12448	2/1507
	Oiartzun	2.80356	37/1311
SVM Car	Laukiz1	16.76686	210/1249
	Laukiz2	40.77700	554/1357
	Luiando	43.80604	661/1507
	Oiartzun	81.23205	1065/1311
XGB Borda	Laukiz1	79.54091	994/1249
	Laukiz2	0.96545	14/1357
	Luiando	66.27871	999/1507
	Oiartzun	41.89759	550/1311

). XGBoost also favored using several hundred features with the exception of Laukiz2, for which only 14 (0.96%) were selected.

3.3. Variable Importance

Permutation-Based Variable Importance

The most important features for datasets HR and VI showed an average decrease in RMSE of 1.06 p.p. (HR, B65) and 1.93 p.p. (VI, Vogelmann2) when permuted (Figure 6). For the HR dataset, most (i.e., six out of ten) relevant features clustered around the infrared region (920–1000 nm), while for VI, eight out of ten concentrate within the wavelength range of 700–750 nm (the so/called “red edge”). For HR, four features in the infrared region (920–1000 nm) were identified by the model to be most important, being associated with a mean decrease in RMSE of around 1 p.p. Overall, apart from the top five features, the vast majority of features showed only a small importance with average decreases in RMSE below 0.5 p.p.

4. Discussion

4.1. Predictive Performance

The best overall performance achieved in this study (SVM with the “Info Gain” filter, RMSE 27.915 p.p.) has to be seen in the light of model overfitting (see Section 4.2). Leaving out the performance on Laukiz2 when aggregating results, the mean RMSE would be around 19 p.p. However, leaving out a single plot would also change the prediction results for the other plots because the observations from Laukiz2 would not be available for model training. Due to the apparent presence of model overfitting in this study, it is suggested that more training data representing a greater variety of situations are needed. A model can only make robust predictions if it has learned relationships across the whole range of the response. Hence, care should be taken when predicting on the landscape scale using models fitted on this dataset due to their lack of generalizability caused by the limitations of the available training data. However, when inspecting the fold/level performances, it can be concluded that the models performed reasonably well, predicting defoliation greater than 50%, but they failed for lower levels. This applied to all learners of this study. In this study, the overall performance across all learners can be classified as “poor” given that only the SVM learner was able to substantially outperform the featureless learner (Table 3). It is worth noting that data quality issues may have affected model performances, as discussed below in detail (Section 4.5).

4.1.1. Model Differences

An interesting finding is the strength of the SVM algorithm when comparing its predictive performance to its competitors (Table 3). These cluster around a performance of 31 p.p., while SVM scored about 3 p.p. better than all other methods. However, we refrain from comparing these results (both relatively and absolute) to other studies since many study design points have an influence on the final result (optimization strategy, data characteristics, feature selection methods, etc.).

A potential limiting factor in this study could be the upper limit of 70 iterations used for the XGBoost algorithm (hyperparameter nrounds), especially for feature sets including NRIs (Table A1). This setting was a compromise between runtime and tuning space extension with the goal to work well for most feature sets. It may be recommendable to increase this upper limit to a value closer to the number of features in the dataset in order to be able to exploit the full potential of this hyperparameter.

4.1.2. Feature Set Differences

One objective of this study was to determine whether expert-based and data-driven feature engineering have a positive influence on model performance. With respect to Figure 3, no overall positive or negative pattern related to specific feature sets was found that would be valid across all models. The performance of RF and XGBoost on the VI feature set was around 4 to 6 p.p. lower than on others. One reason could be the lack of coverage in the wavelength range between 810 nm and 1000 nm (Figure 6). In addition, for all learners but SVM, a better performance was observed when NRI indices were included in the feature set (i.e., NRI-VI, HR-NRI, and HR-NRI-VI).

4.2. Performance vs. Plot Characteristics

The large differences in RMSE obtained on different test folds can be attributed to model overfitting (Table 4). An RMSE of 54.26 p.p. reveals the model’s inability to predict tree defoliation on this plot (Laukiz2). Laukiz2 differs highly in the distribution of the response variable defoliation compared to all other plots (Figure 1). In the prediction scenario for Laukiz2, the model was trained on data containing mostly medium-to-high defoliation values and only few low ones. This caused overfitting on the medium-to-high values, degrading the model’s predictive performance in other scenarios. When Laukiz2 was in the training set, the overall mean RMSE was reduced by up to 50% with single fold performances as good as 9 p.p. RMSE (with Luiando as test set).

There was also no clear pattern in the percentage of features selected based on hyperparameter tuning (Table 7). The optimal value for the number of features (interpreted as a percentage of available features), which are selected from the ranked filter results, is determined by the internal surrogate learner of the MBO tuning method using the results from the previous tuning iterations. Due to this iterative approach, MBO is in some ways able to evaluate how well a learning algorithm plays together with a certain amount of selected features and is subsequently able to adjust the number of variables to an optimal value. In general, considering SVM’s relative success, the use of at least a few hundred features from the combined feature set appears to be beneficial, or at least not harmful when the model’s built-in regularization is capable of dealing with the resulting high-dimensional situation.

Realizing early during hyperparameter optimization that only a few features are needed to reach adequate performances can reduce the overall computational runtime substantially. Hence, regardless of the potential advantage of using filters for increased predictive performance, these can have a strong positive effect on runtime, especially of models making use of hyperparameters that depend on the available number of features, such as RF with $m_{try}$.

Ultimately, the results of Table 7 should be taken with care, as they rely on single model–filter combinations and are subject to random variation. More in-depth research is needed to investigate the effect of filters on criteria other than performance (such as runtime), leading to a multi-criteria optimization problem.

4.3. Feature Selection Methods

The usefulness of filters with respect to predictive performance in this study varied. While the performance of some models (up to 5 p.p. for RF and XGBoost) was improved by specific filters, some models achieved a poorer performance with filters than without them (Figure 4). There was no pattern of specific filters consistently resulting in better scores. Hence, it is recommended to test multiple filters in a study if it is intended to use filters. While filters can improve the performance of models, they may be more appealing based on other aspects than performance. Reducing variables can reduce computational efforts in high-dimensional scenarios and may enhance the interpretability of models. Filters are a lot cheaper to compute than wrapper methods, and the final feature subset selection can be integrated as an additional hyperparameter into the model optimization stage.

Models that used the Borda ensemble method in this study did not perform better on average than models that used a single filter or no filter at all. Ensemble methods have higher stability and robustness than single ones and have shown promising results in [34]. Hence, their expected main advantage is stable performances across datasets with varying characteristics. Single filter methods might yield better model performances on certain datasets but fail on others. The fact that this study used multiple feature sets but only one dataset and tested many single filters could be a potential explanation of why, in all cases, a single filter outperformed the ensemble filter. However, studies that use ensemble filters are still rare, and these are usually not compared against single filters [70]. In summary, in this study, Borda did not perform better than a randomly selected filter method. More case studies applying ensemble filter methods are needed to verify this finding. Nevertheless, ensemble filters can be a promising addition to an ML feature selection portfolio.

PCA, acting as a filter method, more often showed less than optimal results, especially for algorithms RF and XGBoost. XGBoost in particular had substantial problems when using PCA as a filter method and accounted for four of the six worst results (Table 6). However, PCA was able to reduce model fitting times substantially across all algorithms. Depending on the use case, PCA can be an interesting option to reduce dimensionality while keeping runtime low. However, information about the total number of features used by the model is lost when applying this technique. Since filter scores only need to be calculated once for a given dataset in a benchmark setting, the runtime advantage of a PCA vs. filter methods might in fact be negligible in practice.

4.4. Linking Feature Importance to Spectral Characteristics

Unsurprisingly, the most important features for both HR and VI datasets were identified around the red edge of the spectra, specifically in the range of 680 nm to 750 nm.

This area has the highest ability to distinguish between reflectances related to a high density/high foliage density and thus the health status of vegetation and its respective counterpart [71]. However, four out of ten of the most important features of dataset HR are located between 920 nm and 1000 nm. Looking at the spectral curves of the plots, apparent reflectance differences can be observed in this spectral range—especially for plot Oiartzun—which might explain why these features were considered important by the model.

A possible explanation for the poor performances of most models scored on the VI dataset compared to all other feature sets could be the lack of features covering the area between 850 nm and 1000 nm (Figure 6). The majority of VI features covers the range between 550–800 nm. Only one index (PWI) covers information in the range beyond 900 nm.

4.5. Data Quality

Environmental datasets always come with some constraints that can have potential influence on the modeling process and its outcome. Finding a suitable approach to extract the remote sensing information from each tree was a complex process. Due to the reported geometric offset of up to one meter within the hyperspectral data, the risk of assigning a value to an observation that would actually refer to a different, possibly non-tree, pixel was reasonably high. It was concluded that using a buffer radius of one meter can be a good compromise between the inclusion of information from too many surrounding trees and an under-coverage of the tree crown. With the chosen radius, we are confident that we were able to map individual tree crowns while accounting for a possible geometric offset. This results in all cases in four contributing pixels (=four square meters) for the extraction of hyperspectral information for a given tree. Even though no results showing the influence of different buffer values on the extraction were provided, it is hypothesized that the relationships between features would not change substantially, leading to almost identical model results. Instead of using a buffer to extract the hyperspectral information, segmentation could have been considered. However, this method would have required more effort for no clear added value in our view and would have moved the focus of this manuscript more to data preprocessing and away from feature selection methods.

Trees located within grid cells on the border of a plot are a notable exception where the exact number of pixels contributing to the observation’s feature value may be reduced since the image was cropped to the plot’s extent. Cropping was applied to avoid the accidental inclusion of background data such as forest roads. However, this effect was deemed to be of negligible importance.

The available hyperspectral data covered a wavelength between 400 nm and 1000 nm. Hence, the spectral range of the shortwave infrared (SWIR) region is not covered in this study. Given that this range is often used in forest health studies [72], e.g., when calculating the normalized difference moisture index (NDMI) index [73], this marks a clear limitation of the dataset at hand.

The dataset consists of in situ data collected during September 2016, which was matched against remote sensing data acquired at the end of September 2016. A multi-temporal dataset consisting of in situ data from different phenology stages would possibly improve the achieved model performances. However, this would also require the costly acquisition of hyperspectral data of these additional timestamps.

The R package hsdar was used for the calculation of vegetation indices [74]. All indices that could be calculated with the given spectral range of the data (400–1000 nm) were used. This means that even though

Table A2. List of available vegetation indices in the hsdar package.

Name	Formula	Reference
Boochs	$D_{703}$	[98]
Boochs2	$D_{720}$	[98]
CAI	$0.5\times (R_{2000}+R_{2200})-R_{2100}$	[99]
CARI	$a=(R_{700}-R_{550})/150$	[100]
	$b=R_{550}-(a\times 550)$
	$\frac{R_{700}\times |(a\times 670+R_{670}+b)}{R_{670}\times (a^2+1){|}^{0.5}}$
Carter	$R_{695}/R_{420}$	[101]
Carter2	$R_{695}/R_{760}$	[101]
Carter3	$R_{605}/R_{760}$	[101]
Carter4	$R_{710}/R_{760}$	[101]
Carter5	$R_{695}/R_{670}$	[101]
Carter6	$R_{550}$	[101]
CI	$R_{675}\times R_{690}/R_{683}^2$	[102]
CI 2	$R_{760}/R_{700}-1$	[103]
ClAInt	${\int }_{600 \mathrm{nm}}^{735 \mathrm{nm}}R$	[104]
CRI1	$1/R_{515}-1/R_{550}$	[103]
CRI2	$1/R_{515}-1/R_{770}$	[103]
CRI3	$1/R_{515}-1/R_{550}\times R_{770}$	[103]
CRI4	$1/R_{515}-1/R_{700}\times R_{770}$	[103]
D1	$D_{730}/D_{706}$	[102]
D2	$D_{705}/D_{722}$	[102]
Datt	$(R_{850}-R_{710})/(R_{850}-R_{680})$	[105]
Datt2	$R_{850}/R_{710}$	[105]
Datt3	$D_{754}/D_{704}$	[105]
Datt4	$R_{672}/(R_{550}\times R_{708})$	[106]
Datt5	$R_{672}/R_{550}$	[106]
Datt6	$\left(R_{860}\right)/(R_{550}\times R_{708})$	[106]
Datt7	$(R_{860}-R_{2218})/(R_{860}-R_{1928})$	[107]
Datt8	$(R_{860}-R_{1788})/(R_{860}-R_{1928})$	[107]
DD	$(R_{749}-R_{720})-(R_{701}-R_{672})$	[108]
DDn	$2\times (R_{710}-R_{660}-R_{760})$	[109]
DPI	$(D_{688}\ast D_{710})/D_{697}^2$	[102]
DWSI1	$R_{80}/R_{1660}$	[110]
DWSI2	$R_{1660}/R_{550}$	[110]
DWSI3	$R_{1660}/R_{680}$	[110]
DWSI4	$R_{550}/R_{680}$	[110]
DWSI5	$(R_{800}+R_{550})/(R_{1660}+R_{680})$	[110]
EGFN	$\frac{(max\left(D_{650:750}\right)-max\left(D_{500:550}\right))}{(max\left(D_{650:750}\right)+max\left(D_{500:550}\right))}$	[111]
EGFR	$max\left(D_{650:750}\right)/max\left(D_{500:550}\right)$	[111]
EVI	$\frac{2.5\times (R_{800}-R_{670})}{(R_{800}-(6\times R_{670})-(7.5\times R_{475})+1)}$	[112]
GDVI	$(R_{800}^n-R_{680}^n)/(R_{800}^n+R_{680}^n)$	[113]
GI	$R_{554}/R_{677}$	[114]
Gitelson	$1/R_{700}$	[115]
Gitelson2	$(R_{750}-R_{800}/R_{695}-R_{740})-1$	[103]
GMI1	$R_{750}/R_{550}$	[103]
GMI2	$R_{750}/R_{700}$	[103]
Green NDVI	$\frac{R_{800}-R_{550}}{R_{800}+R_{550}}$	[116]
LWVI_1	$\frac{(R_{1094}-R_{983})}{(R_{1094}+R_{983})}$	[117]
LWVI_2	$\frac{R_{1094}-R_{1205}}{R_{1094}+R_{1205}}$	[117]
Maccioni	$\frac{R_{780}-R_{710})}{R_{780}-R_{680}}$	[118]
MCARI	$((R_{700}-R_{670})-0.2\times (R_{700}-R_{550}))\times (R_{700}/R_{670})$	[119]
MCARI2	$((R_{750}-R_{705})-0.2\times (R_{750}-R_{550}))\times (R_{750}/R_{705})$	[120]
mND705	$\frac{(R_{750}-R_{705})}{R_{750}+R_{705}-2\times R_{445}}$	[121]
mNDVI	$\frac{(R_{800}-R_{680})}{R_{800}+R_{680}-2\times R_{445}}$	[121]
MPRI	$\frac{R_{515}-R_{530}}{R_{515}+R_{530}}$	[122]
MSAVI	$0.5\times {({(2\times R_{800}+1)}^2-8\times (R_{800}-R_{670}))}^{0.5}$	[123]
MSI	$\frac{R_{1600}}{R_{817}}$	[124]
mSR	$\frac{R_{800}-R_{445}}{R_{680}-R_{445}}$	[121]
mSR2	$\frac{(R_{750}/R_{705})-1}{R_{750}/R_{705}{+1)}^{0.5}}$	[125]
mSR705	$\frac{R_{750}-R_{445}}{R_{705}-R_{445}}$	[121]
MTCI	$\frac{R_{754}-R_{709}}{R_{709}-R_{681}}$	[126]
MTVI	$1.2\times (1.2\times (R_{800}-R_{550})-2.5\times (R_{670}-R_{550}))$	[127]
NDLI	$\frac{log(1/R_{1754})-log(1/R_{1680})}{log(1/R_{1754})+log(1/R_{1680})}$	[128]
NDNI	$\frac{log(1/R_{1510})-log(1/R_{1680})}{log(1/R_{1510})+log(1/R_{1680})}$	[128]
NDVI	$\frac{R_{800}-R_{680}}{R_{800}+R_{680}}$	[129]
NDVI2	$\frac{R_{750}-R_{705}}{R_{750}+R_{705}}$	[130]
NDVI3	$\frac{R_{682}-R_{553}}{R_{682}+R_{553}}$	[131]
NDWI	$\frac{R_{860}-R_{1240}}{R_{860}+R_{1240}}$	[73]
NPCI	$\frac{R_{680}-R_{430}}{R_{680}+R_{430}}$	[111]
OSAVI	$\frac{(1+0.16)\times (R_{800}-R_{670})}{R_{800}+R_{670}+0.16}$	[132]
OSAVI2	$\frac{(1+0.16)\times (R_{750}-R_{705})}{R_{750}+R_{705}+0.16)}$	[120]
PARS	$\frac{R_{746}}{R_{513}}$	[133]
PRI	$\frac{R_{531}-R_{570}}{R_{531}+R_{570}}$	[134]
PRI_norm	$\frac{PRI\times (-1)}{RDVI\times R_{700}/R_{670}}$	[135]
PRI ∗ CI2	$PRI\ast CI2$	[136]
PSRI	$\frac{R_{678}-R_{500}}{R_{750}}$	[137]
PSSR	$\frac{R_{800}}{R_{635}}$	[138]
PSND	$\frac{R_{800}-R_{470}}{R_{800}-R_{470}}$	[138]
PWI	$\frac{R_{900}}{R_{970}}$	[139]
RDVI	$\frac{R_{800}-R_{670}}{\sqrt{R_{800}+R_{670}}}$	[140]
REP_LE	Red-edge position through linear extrapolation	[141]
REP_Li	$R_{re}=\frac{R_{670}+R_{780}}{2}$	[142]
	$\frac{700+40\times (R_{re}-R_{700})}{(R_{740}-R_{700}))}$
SAVI	$\frac{(1+L)\times (R_{800}-R_{670})}{(R_{800}+R_{670}+L)}$	[143]
SIPI	$\frac{R_{800}-R_{445}}{R_{800}-R_{680}}$	[144]
SPVI	$0.4\times 3.7\times (R_{800}-R_{670})-1.2\times {\left({(R_{530}-R_{670})}^2\right)}^{0.5}$	[145]
SR	$\frac{R_{800}}{R_{680}}$	[146]
SR1	$\frac{R_{750}}{R_{700}}$	[147]
SR2	$\frac{R_{752}}{R_{690}}$	[147]
SR3	$\frac{R_{750}}{R_{550}}$	[147]
SR4	$\frac{R_{700}}{R_{670}}$	[148]
SR5	$\frac{R_{675}}{R_{700}}$	[133]
SR6	$\frac{R_{750}}{R_{710}}$	[149]
SR7	$\frac{R_{440}}{R_{690}}$	[150]
SR8	$\frac{R_{515}}{R_{550}}$	[151]
SRPI	$\frac{R_{430}}{R_{680}}$	[144]
SRWI	$\frac{R_{850}}{R_{1240}}$	[102]
Sum_Dr1	${\sum }_{i=626}^{795}D{1}_i$	[152]
Sum_Dr2	${\sum }_{i=680}^{780}D{1}_i$	[153]
SWIR FI	$\frac{R_{2133}^2}{R_{2225}\times R_{2209}^3}$	[154]
SWIR LI	$3.87\times (R_{2210}-R_{2090})-27.51\times (R_{2280}-R_{2090})-0.2$	[155]
SWIR SI	$-41.59\times (R_{2210}-R_{2090})+1.24\times (R_{2280}-R_{2090})+0.64$	[155]
SWIR VI	$37.72\times (R_{2210}-R_{2090})+6.27\times (R_{2280}-R_{2090})+0.57$	[155]
TCARI	$3\ast ((R_{700}-R_{670})-0.2\times R_{700}-R_{550})\times (R_{700}/R_{670}))$	[127]
TCARI/OSAVI	TCARI/OSAVI	[127]
TCARI2	$3\times ((R_{750}-R_{705})-0.2\times (R_{750}-R_{550})\times (R_{750}/R_{705}))$	[120]
TCARI2/OSAVI2	TCARI2/OSAVI2	[120]
TGI	$-0.5(190(R_{670}-R_{550})-120(R_{670}-R_{480}))$	[156]
TVI	$0.5\times (120\times (R_{750}-R_{550})-200\times (R_{670}-R_{550}))$	[157]
Vogelmann	$\frac{R_{740}}{R_{720}}$	[97]
Vogelmann2	$\frac{R_{734}-R_{747}}{R_{715}+R_{726}}$	[97]
Vogelmann3	$\frac{D_{715}}{D_{705}}$	[97]
Vogelmann4	$\frac{R_{734}-R_{747}}{R_{715}+R_{720}}$	[97]

lists all indices available in the package, not all listed indices were used in this study. Although this selection included a large number of indices, some possibly helpful indices might have been missed due to the restriction of the hyperspectral data.

Overall, the magnitude of uncertainty introduced by the mentioned effects during index derivation cannot be quantified. Such limitations and uncertainties apply to most environmental studies and cannot be completely avoided.

4.6. Practical Implications on Defoliation and Tree Health Mapping

Even though this work has a strong methodological focus by comparing different benchmark settings on highly correlated feature sets, implications on tree health should be briefly discussed in the following. Due to the outlined dataset issues in Section 4.5, which are mainly responsible for the resulting poor model performances, the trained models are not suited for practical use, e.g., to predict defoliation in unknown areas, due to the high mapping uncertainty. However, the general approach of utilizing hyperspectral data to classify the health status of trees partly led to promising results. For example, due to the narrow bandwidth of the hyperspectral sensor, small parts of the spectrum (mainly in the infrared region) were of higher importance to the models (e.g., see Figure 6), meaning that they helped the models to distinguish between low and high tree defoliation. If spatial offset errors of the image data and possible background noise can be reduced (possibly by making use of image segmentation), we believe that model performances could be substantially enhanced. Such improved models, starting around an RMSE of 20% and less, should be able to provide added value to support the long-term monitoring of forest health and early detection of fungi-affected tree plots. Nevertheless, overall the use of defoliation as a proxy for forest health should be critically reconsidered as it comes with various practical issues, starting from potential offsets during data collection, varying leaf density due to tree age, and differing effects between tree species, to name just a few.

4.7. Comparison to Other Studies

While most defoliation studies operate on the plot level using coarser resolution multispectral satellite data [10,75,76], there are also several recent studies using airborne or ground-based sensors at the tree level. Among these, refs. [77,78] used ground-level methods, such as airborne laser scanning (ALS) and light detection and ranging (LiDAR).

Studies focusing on tree-level defoliation mainly used ground-level methods, such as ALS or LiDAR [77,78]. Ref. [77] used ordinary least squares (OLS) regression methods while [78] retrieving information from ground-level RGB photos using convolutional neural networks (CNN). However, neither of them used spatial CV for model assessment, and [78] did not perform feature selection (FS). The authors of [79] used a partial least squares (PLS) model with high-resolution digital aerial photogrammetry (DAP) to predict cumulative defoliation caused by the spruce budworm. Study results indicated that spectral features were found to be most helpful for the model. Incorporating such features (both spectral and structural) could be a possible enhancement for future works. No studies were found to model defoliation caused by Diplodia sapinea (Fr.) Fuckel with remote sensing data, and most studies focused on describing the tree conditions based on local sampling [80,81].

The field of (hyperspectral) remote sensing has had a strong focus on using RF for modeling in recent years [82]. However, in high-dimensional scenarios, tuning the parameter $m_{try}$ becomes computationally expensive. To account for this and the high dimensionality in general, studies used feature selection approaches, such as semi-supervised feature extraction [83], wrapper methods [84,85,86], PCA, and adjusted feature selection [87]. In general, applying feature selection methods on hyperspectral datasets has shown to be effective, regardless of the method used [88,89]. However, no studies were found that made explicit use of filter methods in combination with hyperparameter tuning in the field of (hyperspectral) remote sensing. Potential reasons for this absence could be an easier programmatic access to wrapper methods and a higher general awareness of these compared to filter methods. Applying the filter-based feature selection methodology shown in this study and its related code provided in the research compendium might be a helpful reference for future studies using hyperspectral remote sensing data.

When looking for remote sensing studies that compare multiple models, it turned out that these often operate in a low-dimensional predictor space [90] or use wrapper methods explicitly [86].

Refs. [91,92] are more similar in their methodology but focus on a different response variable (woody cover). Ref. [91] used machine learning with ALS data to study dieback of trees in eucalyptus forests. A grid search was used for hyperparameter tuning and forward feature selection (FFS) for variable selection. Ref. [92] analyzed woody cover in South Africa using a spatial CV and FS approach [93] with an RF classifier. Ref. [94] shows a similar setup; they used hyperspectral vegetation indices and a nested CV approach for performance estimation, and they estimated variable importance targeting woody biomass as the response. In the results, lasso showed the best performance among the chosen methods. However, the authors did not optimize the hyperparameters of RF, which makes a fair comparison problematic since the other models used internal optimization. The discussion section of [94] lists additional studies that made use of shrinkage models for high-dimensional remote sensing modeling.

In summary, no studies could be found that used filter methods for FS or made use of NRI indices in their work and had a relation to tree health. This might relate to the fact that most environmental datasets are not high dimensional. In fact, many studies use fewer than ten features, and issues related to correlations are often solved manually instead of relying on an automated approach. These manual approaches might suffer from subjectivity and may limit the reproducibility of results.

Other fields (e.g., bioinformatics) encounter high-dimensional datasets more often. Hence, more studies using (filter-based) feature selection approaches can be found in this field [95,96]. However, bioinformatics differs conceptually in many ways from environmental modeling, and, therefore, no greater focus was put into comparing studies of this field. The availability of high-dimensional feature sets will increase in the future due to higher temporal and spectral resolutions of sensors. In addition, a high-spatial resolution comes with the possibility of calculating many textural features. Hence, the ability to deal with high-dimensional datasets becomes more important, and unbiased robust approaches are needed. We hope that this work and its methodology raise awareness about the application of filter methods to tackle high-dimensional problems in the environmental modeling field.

5. Conclusions

This study analyzed the effectiveness of filter-based feature selection in improving various machine-learning models of defoliation of trees in northern Spain based on hyperspectral remote-sensing data. Substantial differences in performance occurred depending on which feature selection and machine learning methods were combined. SVM showed the most robust behavior across all highly correlated datasets and was able to predict the response variable of this study substantially better than other methods.

Filter methods were able to improve the predictive performance on datasets in some instances, although there was no clear and systematic pattern. Their effectiveness depends on the algorithm and the dataset characteristics. Ensemble filter methods did not show a substantial improvement over individual filter methods in this study.

The addition of derived feature sets was, in most cases, able to improve predictive performance. In contrast, feature sets that focused on only a small fraction of the available spectral range (i.e., dataset VI) showed a worse performance than the ones that covered a wider range (400–1000 nm; HR, NRI). NRIs can be seen as a valuable addition to the optimization of predictive performance in the remote sensing of vegetation.

Features along the red-edge wavelength region were most important for models during prediction. With respect to dedicated vegetation indices, all versions of the Vogelmann index were seen as the most important indices for the best performing SVM model. This matches well with the actual purpose of these indices—they were invented to detect defoliation on sugar maple trees (Acer saccharum Marsh.) caused by pear thrips (Taeniothrips inconsequens Uzel) [97]. However, assessing feature importance for highly correlated features remains a challenging task. Results might be biased and should be taken with care to avoid overgeneralizing from individual studies.

Finally, the potential of predicting defoliation with the given study design was rather limited with respect to the average RMSE of 28 p.p. scored by the best performing model. More training data covering a wider range of defoliation values in a larger number of forest plantations are needed to train better models that can create more robust predictions.