Combination of Sentinel-1 and Sentinel-2 Data for Tree Species Classification in a Central European Biosphere Reserve

Abstract

Microwave and optical imaging methods react differently to different land surface parameters and, thus, provide highly complementary information. However, the contribution of individual features from these two domains of the electromagnetic spectrum for tree species classification is still unclear. For large-scale forest assessments, it is moreover important to better understand the domain-specific limitations of the two sensor families, such as the impact of cloudiness and low signal-to-noise-ratio, respectively. In this study, seven deciduous and five coniferous tree species of the Austrian Biosphere Reserve Wienerwald (105,000 ha) were classified using Breiman’s random forest classifier, labeled with help of forest enterprise data. In nine test cases, variations of Sentinel-1 and Sentinel-2 imagery were passed to the classifier to evaluate their respective contributions. By solely using a high number of Sentinel-2 scenes well spread over the growing season, an overall accuracy of 83.2% was achieved. With ample Sentinel-2 scenes available, the additional use of Sentinel-1 data improved the results by 0.5 percentage points. This changed when only a single Sentinel-2 scene was supposedly available. In this case, the full set of Sentinel-1-derived features increased the overall accuracy on average by 4.7 percentage points. The same level of accuracy could be obtained using three Sentinel-2 scenes spread over the vegetation period. On the other hand, the sole use of Sentinel-1 including phenological indicators and additional features derived from the time series did not yield satisfactory overall classification accuracies (55.7%), as only coniferous species were well separated.

1. Introduction

The ongoing species loss and the continued degradation of many terrestrial ecosystems make it increasingly important to monitor changes on the Earth’s surface on a large scale, with high accuracy and low latency [1]. The multispectral image data generated by the Sentinel-2 (S2) twin satellites are provided free of charge through the European Copernicus program. These data provide a great opportunity to monitor the entire Earth’s surface with high spatial and spectral as well as temporal resolution [2,3]. Several studies have already shown that the use of multispectral imagery generates highly informative data for land cover and tree species classification [4,5]. Further improvements in classification accuracy can be achieved by using multispectral time series [6,7,8].

While the high 5-day-temporal resolution of the S2 satellites leads to dense time series, it is not guaranteed that areas larger than 10³–10⁴ km² are fully covered by the very same cloud-free acquisitions. The selection of suitable image material is, therefore, often easy and straightforward for smaller areas, but for large areas, additional pre-processing steps are needed to ensure a homogenous and gap-free set of features. Suitable techniques are, for example, compositing techniques [9], gap-filling procedures [10,11], or the use of descriptive temporal metrics [12].

Radar sensors, on the other hand, provide a more continuous data stream with however a lower signal-to-noise-ratio (SNR), as well as terrain- and observation-geometry-related artifacts [13,14]. The two Sentinel-1 satellites (S1), which possess high spatial resolution and high revisit frequency, are also provided free of charge by the Copernicus program and generate microwave images under almost all weather conditions. A number of studies have demonstrated a good potential for the differentiation of deciduous trees and conifers [14,15]. Rüetschi et al. [16] obtained an overall accuracy of 72% for a test site in Switzerland. Udali et al. [17] presented a forest type and tree species classification in a test area in southern Sweden using multitemporal S1 data with overall accuracies of 94% and 66%, respectively.

The combination of S1 and S2 data was used in several studies [7,15] for forest type and tree species classifications. Bjerreskov et al. [7] used a combination of multitemporal S1 and S2 data to classify nemoral forests in Denmark into broadleaf and coniferous forest types as well as into predefined tree species groups with overall accuracies of 95% and 63%, respectively. A combination of S1 and optical Landsat imagery was used for the classification of dominant tree species in broadleaf deciduous forests in Vietnam with an overall accuracy of 79% [18]. Systematic ablation studies are lacking investigating the potential of the two electromagnetic domains for the classification of a higher number (≥10) of tree species in forests with high diversity.

The objective of this paper is to study the benefits of combining S1 and S2 data for tree species classification in the mid-altitude forests of Austria. Various S1 and S2 data combinations are used to classify 12 different tree species with the help of Breiman’s random forest classifier [19] to evaluate the discriminative power of the two data streams:

(1)

a set of S1-derived parameters and 14 cloud-free S2 scenes were classified individually and in combination;

(2)

the monotemporal S2 scenes were classified separately as well as paired with S1 data;

(3)

the accuracies obtained from the monotemporal S2 scenes were used to determine the most- and least-accurate S2 scenes from spring, summer, and autumn seasons. Combinations of the least- and most-accurate seasonal S2 scenes were classified with and without S1 data.

2. Materials and Methods

2.1. Study Site, Reference Data

The Biosphere Reserve Wienerwald (BPWW) is located southwest of Vienna (Austria) and covers an area of approximately 105,000 hectares with a geographical extension of ca. 42 km × 47 km and an elevation between 162 and 893 m above sea level. The broadleaf-dominated forest is characterized by more than 20 forest communities [20]. Due to its diversity of tree species and its location within the overlapping area of two S2 orbits, it is particularly well suited for investigating remote sensing methods.

The study site and the reference data are shown in Figure 1. An existing dataset, created by using information from several forest enterprises such as forest inventories and stand-wise description of the forest management plans, was used as reference data [6]. Additional samples were added to better balance the different classes. While it was not always possible for sparsely represented tree species, a maximum of one pixel per forest stand was selected. The final dataset consisted of 1283 individual pixels, representing a total of 12 tree species—seven deciduous and five coniferous. Although not all tree species occurring in the BPWW were represented in the samples, they nevertheless provided a good overview of the main tree species prevailing in the park.

2.2. Sentinel-2 Data

All 14 completely cloud-free S2 scenes from the 2018 growing period (April to October) were selected (

Table 1. Summary of the Sentinel-2 (S2) scenes of the 2018 vegetation period used for classification.

S2 Satellite	Date	Orbit	Sun Zenith Angle	Sun Azimuth Angle
B	8 April 2018	79	43.02	157.29
B	21 April 2018	122	37.72	160.39
A	6 May 2018	122	33.06	159.37
A	2 July 2018	79	28.23	147.73
B	9 August 2018	122	34.38	155.97
A	21 August 2018	79	38.49	154.81
B	29 August 2018	122	40.40	160.44
A	13 September 2018	122	45.59	163.93
B	18 September 2018	122	47.41	164.98
B	28 September 2018	122	51.10	166.96
A	30 September 2018	79	52.25	164.21
B	5 October 2018	79	54.08	165.12
A	10 October 2018	79	55.90	165.94
A	30 October 2018	79	62.82	168.14

). Since Nkosi et al. [21] found band 9 to have a high capability in discriminating tree species, band 9 was left in the dataset, while bands 1 and 10 were excluded. The remaining eleven S2 spectral bands were resampled to a unique 10 m spatial resolution and corrected using the Sen2Cor atmospheric correction [22]. The resulting dataset was expanded by two biophysical vegetation variables calculated from visible and NIR spectral channels: FAPAR and the LAI [23,24]. In addition, 30 vegetation indices were calculated and added to the dataset (

Table A1. Summary of the additional vegetation indices used for the classification, together with the corresponding formula and references (band 8 was used for the NIR = Near-Infrared; RE = Red-Edge).

Index-Name	Formula	Reference
Built-up Area Index (BAI)	$\frac{\mathrm{BLUE}-\mathrm{NIR}}{\mathrm{BLUE}+\mathrm{NIR}}$	[49]
Chlorophyll Green Index (CGI)	$\frac{\mathrm{NIR}}{\mathrm{GREEN}+\mathrm{RE}1}$	[50]
Greenness Index (GI)	$\frac{\mathrm{GREEN}}{\mathrm{RED}}$	[51]
Green Normalized-Difference Vegetation Index (gNDVI)	$\frac{\mathrm{NIR}-\mathrm{GREEN}}{\mathrm{NIR}+\mathrm{GREEN}}$	[52]
Leaf Chlorophyll Content Index (LCCI)	$\frac{\mathrm{RE}3}{\mathrm{RE}1}$	[53]
Moisture Stress Index (MSI)	$\frac{\mathrm{SWIR}1}{\mathrm{NIR}}$	[54]
Normalized-Difference Red-Edge and SWIR2(NDRESWIR)	$\frac{\mathrm{RE}2-\mathrm{SWIR}2}{\mathrm{RE}2+\mathrm{SWIR}2}$	[55]
Normalized-Difference Tillage Index (NDTI)	$\frac{\mathrm{SWIR}1-\mathrm{SWIR}2}{\mathrm{SWIR}1+\mathrm{SWIR}2}$	[56]
Normalized-Difference Vegetation Index (NDVI)	$\frac{\mathrm{NIR}-\mathrm{RED}}{\mathrm{NIR}+\mathrm{RED}}$	[57]
Red-Edge Normalized-Difference Vegetation Index(reNDVI)	$\frac{\mathrm{NIR}-\mathrm{RE}1}{\mathrm{NIR}+\mathrm{RE}1}$	[52]
Normalized-Difference Water Index 1 (NDWI1)	$\frac{\mathrm{NIR}-\mathrm{SWIR}1}{\mathrm{NIR}+\mathrm{SWIR}1}$	[58]
Normalized-Difference Water Index 2 (NDWI2)	$\frac{\mathrm{NIR}-\mathrm{SWIR}2}{\mathrm{NIR}+\mathrm{SWIR}2}$	[52]
Normalized Humidity Index (NHI)	$\frac{\mathrm{SWIR}2-\mathrm{GREEN}}{\mathrm{SWIR}2+\mathrm{GREEN}}$	[59]
Red-Edge Peak Area (REPA)	$\mathrm{RED}+\mathrm{RE}1+\mathrm{RE}2+\mathrm{RE}3+\mathrm{NIR}$	[55,60]
Red SWIR1 Difference (DIRESWIR)	$\mathrm{RED}+\mathrm{SWIR}1$	[61]
Red-Edge Triangular Vegetation Index (RETVI)	$100\left(\mathrm{NIR}-\mathrm{RE}1\right)-10\left(\mathrm{NIR}-\mathrm{GREEN}\right)$	[62]
Soil Adjusted Vegetation Index (SAVI)	$\frac{\mathrm{NIR}-\mathrm{RED}}{\mathrm{NIR}+\mathrm{GREEN}+0.5}1.5$	[63]
Blue and RE1 Ratio (SRBRE1)	$\frac{\mathrm{BLUE}}{\mathrm{RE}1}$	[51]
Blue and RE2 Ratio (SRBRE2)	$\frac{\mathrm{BLUE}}{\mathrm{RE}2}$	[64]
Blue and RE3 Ratio (SRBRE3)	$\frac{\mathrm{BLUE}}{\mathrm{RE}3}$	[55]
NIR and Blue Ratio (SRNIRB)	$\frac{\mathrm{NIR}}{\mathrm{BLUE}}$	[65]
NIR and Green Ratio (SRNIRG)	$\frac{\mathrm{NIR}}{\mathrm{GREEN}}$	[51]
NIR and Red Ratio (SRNIRR)	$\frac{\mathrm{NIR}}{\mathrm{RED}}$	[65]
NIR and RE1 Ratio (SRNIRRE1)	$\frac{\mathrm{NIR}}{\mathrm{RE}1}$	[50]
NIR and RE2 Ratio (SRNIRRE2)	$\frac{\mathrm{NIR}}{\mathrm{RE}2}$	[55]
NIR and RE3 Ratio (SRNIRRE3)	$\frac{\mathrm{NIR}}{\mathrm{RE}3}$	[55]
Soil Tillage Index (STI)	$\frac{\mathrm{SWIR}1}{\mathrm{SWIR}2}$	[56]
Water Body Index (WBI)	$\frac{\mathrm{BLUE}-\mathrm{RED}}{\mathrm{BLUE}+\mathrm{RED}}$	[66]

).

2.3. Sentinel-1 Data

In this study, 250 S1 ground-range-detected (GRD) interferometric wide (IW) swath mode acquisitions from the year 2018 were used. The pre-processed data were available via the Austrian Data Cube [25]. The pre-processing steps included precise orbit correction, border noise removal, radiometric correction to ß⁰ values, radiometric terrain flattening, range-Doppler terrain correction, and conversion to the decibel scale. A terrain model based on airborne laser scanning resampled onto a 10 m grid was used for the radiometric terrain flattening and the range-Doppler terrain correction steps. From the multitemporal S1 data, several parameters were computed. These included temporally averaged backscatter values for given time periods, phenological parameters, and harmonic regression model parameters.

2.3.1. Backscatter Averages and Ratios

For each repeat cycle of the S1 satellites (12 days), the temporal average of S1 backscatter was computed (

Table 2. Summary of the Sentinel-1 temporal average backscatter parameters of the 2018 vegetation period used for classification.

Temporal Average Backscatter	Leaf-Off	Leaf-On
VH	20180314_20180326_VH	20180618_20180630_VH
VV	20180314_20180326_VV	20180618_20180630_VV
VH/VV		20180618_20180630_CPR

). Two values per polarization (VV and VH)—one representing snow free, leaf-off conditions (14 to 26 March 2018) and one representing leaf-on conditions (18 to 30 June 2018) were selected. Furthermore, in case of the leaf-on period, the cross ratio (CPR) was computed as the backscatter ratio between VH and VV polarization [26,27], and in the case of VH polarization, the backscatter ratio between the leaf-on (18 to 30 June 2018) and leaf-off (14 to 26 March 2018) conditions was included.

2.3.2. Phenological Parameters

Deciduous forest classes show distinct backscatter behavior where the VH backscatter drops during the summer period by 1–2 dB when compared to the leaf-off period [16,28]. Several studies assumed that the drop in backscatter is connected to the leaf emergence, while the backscatter increase is caused by the leaf fall [16,28,29,30]. We applied the breakpoints algorithm described in Zeileis et al. [31] and successfully tested on the annual time series of the 12-day VH backscatter averages [16]. The computation was limited to pixels, where the average backscatter in the leaf-on period was lower than that in the leaf-off period (hence the ratio between the temporally averaged backscatter for leaf-on and leaf-off conditions was positive). The first breakpoint in the time series was assumed to represent the start of the season, while the second breakpoint represented the end of the season. The length of the season was computed as the difference between the two values. Start of season, end of season, and length of season were used in this study (

Table 3. Summary of the Sentinel-1 phenological parameters of the 2018 vegetation period used for classification.

Backscatter ratio	VH or VV
Leaf-on/Leaf-off	Rat_Leaf_on_off
Phenology	VH or VV
Start of season—day of the year	sos_doy
End of season—day of the year	eos_doy
Length of season—days	sos_doy
	correlation_winter
	slope_winter

).

2.3.3. Harmonic Parameters

Changes in the vegetation structure and environmental conditions cause temporal changes in radar backscatter. Especially in the case of vegetation, these changes typically have a strong seasonal character. Harmonic models (Equation (1)) can be used to describe this seasonal backscatter variation [32].

$${\widehat{\gamma }}_{t_{day}}^0=\overline{{\gamma }^0}+{\displaystyle \sum }_{i=1}^k\left(C_i\cos \frac{2\pi i{t}_{day}}{n}+S_i\sin \frac{2\pi i{t}_{day}}{n}\right)$$

(1)

The model estimates the most probable radar backscatter, ${\widehat{\gamma }}_{t_{day}}^0,$ for a given day of the year, $t_{day},$ from the average backscatter, $\overline{{\gamma }^0}$, for the given time period (year 2018 in case of this study) and the harmonic coefficients of the cosine and sine components, $C_i$ and $S_i$. Harmonic coefficients and average backscatter are referred to as harmonic parameters (HPAR). As suggested, k is set to 3, representing the processes of a time scale of four months [32].

The HPARs (

Table 4. Summary of the Sentinel-1 harmonic parameters of the 2018 vegetation period used for classification.

		Ascending (Orbit 73)	Descending (Orbit 22)
HPAR
Cosine 1	VH	HPAR-C1_2018_VH_A073	HPAR-C1_2018_VH_D022
	VV	HPAR-C1_2018_VV_A073	HPAR-C1_2018_VV_D022
Cosine 2	VH	HPAR-C2_2018_VH_A073	HPAR-C2_2018_VH_D022
	VV	HPAR-C2_2018_VV_A073	HPAR-C2_2018_VV_D022
Cosine 3	VH	HPAR-C3_2018_VH_A073	HPAR-C3_2018_VH_D022
	VV	HPAR-C3_2018_VV_A073	HPAR-C3_2018_VV_D022
Sine 1	VH	HPAR-S1_2018_VH_A073	HPAR-S1_2018_VH_D022
	VV	HPAR-S1_2018_VV_A073	HPAR-S1_2018_VV_D022
Sine 2	VH	HPAR-S2_2018_VH_A073	HPAR-S2_2018_VH_D022
	VV	HPAR-S2_2018_VV_A073	HPAR-S2_2018_VV_D022
Sine 3	VH	HPAR-S3_2018_VH_A073	HPAR-S3_2018_VH_D022
	VV	HPAR-S3_2018_VV_A073	HPAR-S3_2018_VV_D022
HPAR temporal average
	VH	HPAR-M0_2018_VH_A073	HPAR-M0_2018_VH_D022
	VV	HPAR-M0_2018_VV_A073	HPAR-M0_2018_VV_D022
HPAR model error
	VH	HPAR-STD_2018_VH_A073	HPAR-STD_2018_VH_D022
	VV	HPAR-STD_2018_VV_A073	HPAR-STD_2018_VV_D022

) are derived from a least-squares estimation based on the backscatter values and corresponding observation times of the input S1 time series. As opposed to Schlaffer et al. [32], we used the backscatter observations directly instead of using 10-day composites. Due to the strong dependency of the backscatter on the acquisition geometry, the parameter estimation was performed separately for each unique acquisition geometry (e.g., relative S1 orbit). As an additional parameter, the standard deviation of the residual error of the harmonic model was calculated as the square root of the sum of squared errors (SSE), divided by the number of data points (N_points) adjusted for the degrees of freedom of the model (Equation (2)). The SSE was derived from the pixel’s backscatter time-series ${\gamma }_{t, r}^0$ and its harmonic model ${\widehat{\gamma }}_{t, r}^0$.

For this study, we computed HPARs for both VV and VH polarization and for one ascending (relative orbit number 73) and one descending (relative orbit number 22) orbit.

$$s=\sqrt{\frac{SSE\left({\sigma }_{t, r}^0, {\widehat{\sigma }}_{t, r}^0\right) }{N_{points}-2}}$$

(2)

2.4. Classification Approach

Different test cases were defined (

Table 5. Scenario of the nine test cases evaluated using different combinations of Sentinel-1 (S1) and Sentinel-2 (S2) data.

Test Case	Acronym	Features	Comment
1	S1	43	multitemporal S1 parameters
2	S2 (MULTI)	574	multitemporal image data of S2
3	S1 + S2 (MULTI)	617	multitemporal S1 parameters + multitemporal image data of S2
4	S2 (MONO)	41	monotemporal image data of S2
5	S1 + S2 (MONO)	84	multitemporal S1 parameters + monotemporal image data of S2
6	S2 (MAS)	123	image data of Most Accurate S2 Scene of each growing season
7	S1 + S2 (MAS)	166	multitemporal S1 parameters + image data of Most Accurate S2 Scene of each growing season
8	S2 (LAS)	123	image data of Least Accurate S2 Scene of each growing season
9	S1 + S2 (LAS)	166	multitemporal S1 parameters + image data of Least Accurate S2 Scene of each growing season

), which included various band and parameter combinations of the two satellite systems.

Test case 1 to Test case 3 represent various combinations of the respective total data of the two satellite systems. While Test case 4 and Test case 5 were defined to evaluate the influence of additional S1 data on a monotemporal S2 scene, they were also necessary to identify the S2 scenes to be used in Test case 6 to Test case 9. Test cases 6–9 used a selection of the three scenes from the spring, summer, and autumn seasons, which had the highest/lowest total accuracy. Of these, classification models were created as the “most accurate scene of season” (MAS) and the “least accurate scene of season” (LAS) with and without the S1 features, respectively.

These 9 test cases (Table 5) were passed to Breiman’s [19] random forest (RF) algorithm, a widely used ensemble learning approach. To avoid overfitting, a recursive “mean decrease in accuracy” feature selection (MDA) was performed similarly to other studies [6,33,34,35]. The number of trees in the random forests was chosen with ntree = 1000, and for mtry (number of predictors randomly sampled for each node), the default value was used, that is the square root of available input variables. The accuracy assessment was done based on the out-of-bag-results (OOB) of the random forest models calculating common metrics based on confusion matrices.

3. Results

3.1. Full S1/S2 Dataset Comparison

In

Table 6. Confusion matrix based on the out-of-bag-result of Test case 1, using only Sentinel-1 data. UA: user’s accuracy, PA: producer’s accuracy, OA: overall accuracy; for the abbreviations of tree species names see Figure 1.

			Reference
			FS	AG	FE	QU	PR	CP	AC	PA	PN	PS	LD	PM	UA	F1-Score
Sentinel-1 − Test case 1	Classification	FS	236	33	36	86	16	39	34	0	10	4	14	0	46.5%	0.584
		AG	0	0	0	1	0	1	0	0	0	0	0	1	NA	NA
		FE	3	2	6	2	1	1	4	0	0	0	0	0	31.6%	0.113
		QU	47	14	40	140	2	18	13	0	4	1	0	0	50.2%	0.550
		PR	1	0	0	0	1	0	0	0	0	0	0	0	50.0%	0.074
		CP	0	0	2	0	0	4	0	1	0	0	0	1	50.0%	0.101
		AC	0	0	0	0	0	0	0	0	0	0	0	0	NA	NA
		PA	1	0	0	0	0	2	0	119	2	7	0	24	76.8%	0.821
		PN	7	2	2	1	0	5	2	1	114	19	1	5	71.7%	0.755
		PS	0	0	0	0	1	0	0	5	11	44	1	5	65.7%	0.603
		LD	5	1	1	0	4	1	2	1	0	2	31	1	63.3%	0.626
		PM	0	0	0	0	0	0	0	8	2	2	3	19	55.9%	0.422
∑ Reference			300	52	87	230	25	71	55	135	143	79	50	56
PA			78.7%	0.0%	6.9%	60.9%	4.0%	5.6%	0.0%	88.1%	79.7%	55.7%	62.0%	33.9%
									OA 55.7%			Kappa 0.469

, the results of the model created exclusively with S1 data are shown (Test case 1). An overall accuracy (OA) of 55.7%, with a Cohen’s kappa of 0.469, was achieved. While good class-specific accuracies were achieved, especially for the conifers, the deciduous trees, apart from European beech (Fagus sylvatica FS), could only be separated with significantly lower accuracy. Furthermore, not a single sample of the two tree species maple (Acer spp. AC) and alder (Alnus glutinosa AG) could be assigned to the correct class.

If the sample classes were stratified in broadleaved (BL) and coniferous (CO) groups, the random forest model was able to separate them very well (BL = 96.5%, CO = 92.7%).

In the model built using only the S2 data (Test case 2), the overall accuracy was 83.2% with a Cohen’s kappa of 0.806. We also observed an increase in overall accuracy between the broadleaved and coniferous groups (BL = 99.4%, CO = 97.2%) compared to Test case 1, while a significant increase occurred in both the user’s and producer’s accuracies of all tree species. The two species that could not be classified using the S1 data were also relatively well separated (

Table 7. Confusion matrix based on the out-of-bag-result of Test case 2, using only Sentinel-2 data. UA: user’s accuracy, PA: producer’s accuracy, OA: overall accuracy; for the abbreviations of tree species names see Figure 1.

			Reference
			FS	AG	FE	QU	PR	CP	AC	PA	PN	PS	LD	PM	UA	F1-Score
Sentinel-2 − Test case 2	Classification	FS	272	4	10	26	5	22	15	0	2	0	6	0	75.1%	0.822
		AG	1	42	3	1	1	1	0	0	0	0	0	0	85.7%	0.832
		FE	6	2	64	5	2	4	3	0	2	0	2	0	71.1%	0.723
		QU	13	1	7	195	5	4	4	0	0	0	1	0	84.8%	0.848
		PR	0	0	0	0	12	0	0	0	0	0	0	0	100.0%	0.649
		CP	3	2	1	1	0	40	1	0	0	0	0	0	83.3%	0.672
		AC	5	1	0	1	0	0	30	0	0	0	0	0	81.1%	0.652
		PA	0	0	0	0	0	0	0	127	1	4	1	3	93.4%	0.937
		PN	0	0	0	0	0	0	2	3	133	1	1	1	94.3%	0.937
		PS	0	0	0	0	0	0	0	3	3	72	3	5	83.7%	0.873
		LD	0	0	2	1	0	0	0	0	2	2	36	2	80.0%	0.758
		PM	0	0	0	0	0	0	0	2	0	0	0	45	95.7%	0.874
∑ Reference			300	52	87	230	25	71	55	135	143	79	50	56
PA			90.7%	80.8%	73.6%	84.8%	48.0%	56.3%	54.5%	94.1%	93.0%	91.1%	72.0%	80.4%
									OA 83.2%			Kappa 0.806

).

The results of Test case 3, combining all data, show a marginal 0.5-percentage-point improvement in overall accuracy with OA = 83.7 % and kappa = 0.811, compared to Test case 2 (

Table 8. Confusion matrix based on the out-of-bag-result of Test case 3, using Sentinel-1 data and Sentinel-2 data. UA: user’s accuracy, PA: producer’s accuracy, OA: overall accuracy; for the abbreviations of tree species names see Figure 1.

			Reference
			FS	AG	FE	QU	PR	CP	AC	PA	PN	PS	LD	PM	UA	F1-Score
Sentinel-1 + Sentinel-2 − Test case 3	Classification	FS	271	2	9	28	8	19	16	0	1	0	3	0	75.9%	0.825
		AG	0	43	1	0	0	1	0	0	0	0	0	0	95.6%	0.887
		FE	6	2	64	6	0	3	4	0	1	0	2	0	72.7%	0.731
		QU	16	1	9	193	6	4	3	0	0	0	0	0	83.2%	0.835
		PR	0	0	0	0	10	0	0	0	0	0	0	0	100%	0.571
		CP	2	3	3	2	0	44	1	0	1	0	0	0	78.6%	0.693
		AC	4	1	0	0	1	0	29	0	0	0	0	0	82.9%	0.644
		PA	0	0	0	0	0	0	0	129	1	4	1	5	92.1%	0.938
		PN	1	0	0	0	0	0	2	3	135	2	2	0	93.1%	0.938
		PS	0	0	0	0	0	0	0	2	3	70	2	3	87.5%	0.881
		LD	0	0	1	1	0	0	0	0	1	3	40	2	83.3%	0.816
		PM	0	0	0	0	0	0	0	1	0	0	0	46	97.9%	0.893
∑ Reference			300	52	87	230	25	71	55	135	143	79	50	56
PA			90.3%	82.7%	73.6%	83.9%	40.0%	62.0%	52.7%	95.6%	94.4%	88.6%	80.0%	82.1%
									OA 83.7%			Kappa 0.811

). The within-coniferous-group accuracy slightly gained by 1.1 percentage points compared to that for the sole use of optical data (Test case 2). The results of the individual classes remained more or less constant.

To better understand the inputs, Figure 2 lists the 15 most important out of the 50 remaining variables after the MDA-Feature selection of Test case 3. The analysis reveals that, with “20180618_20180630_VH” and “20180618_20180630_CPR” (red), only two S1 parameters were among the 50 remaining variables of the model. Interestingly, they were in the first and third place of importance. Band 9 was not included in the remaining variables.

3.2. Added Value of S1 on Monotemporal S2 Datasets

To evaluate the contributions of S1 data in the (extreme) case of only one available S2 scene, the individual S2 scenes were each classified separately with (Test case 5) and without S1 data (Test case 4). Figure 3 compares the two variants for each of the 14 S2 acquisitions of Test case 4 and Test case 5, each with (blue) and without S1 data (green). Furthermore, as surrogates for compositing approaches, the three most-accurate (MAS) and the three least-accurate scenes (LAS) of each growing season are shown, which serve as S2 data material for Test case 6 to Test case 9. The last bar shows the result of using all available S2 data, with (blue) and without S1 data (green).

The monotemporal S2 results show an increase of the OA from spring to summer and again slightly lower values in the fall. Adding S1 data, the overall accuracy increased by around 5 percentage points but still fell far short of the accuracies achieved in Test case 2 (all S2 scenes without any S1; Figure 3 right, green). Each of the 14 monotemporal S2 scenes largely outperformed (by 5.2 to 14.3 percentage points) the full S1 dataset (horizontal red line in Figure 3).

3.3. Added Value of S1 on Multitemporal S2 Dataset

The two variants of seasonal S2 data (LAS vs. MAS) were separated by roughly 10 percentage points from each other (71.1% vs. 80.0%) with very minor improvements when the suite of S1 features were included (Figure 3). The LAS variant without S1 data, in which for each of the three seasons only the worst performing S2 image was retained (Test case 8), still performed better than the single best performing S2 scene (July image) and largely better (15.4%) compared to the full S1 dataset. The differences between S1 and S2 data were further accentuated (by almost 9%) when the three seasonal images were composed of the best performing individual S2 images (MAS; Test case 6). For this test case, classification results were almost on par with Test case 2 (all S2 scenes) and Test case 3 (all S1 and S2 data).

When comparing the sample-class F₁-scores of Test cases 5 to Test case 9, shown in Figure 4a, it becomes apparent that already three S2 MASs were sufficient to eliminate the added value of S1 data on the classification result. If only three S2 LASs were available, there was a marginal improvement in the F₁-scores of individual sample classes, but the results of already well-classified sample-classes were not further improved.

Nevertheless, it was not possible to reach the highest overall accuracy of Test case 3 by using only three individual S2 scenes, with and without additional S1 data.

4. Discussion

The classification based only on S1 data (Test case 1) did not achieve the same high accuracy values as the classification with either monotemporal and/or combined S2 image data. Nevertheless, it was already possible to separate the five coniferous species with a moderate degree of accuracy. This underlines the potential of S1 for separating different conifers. Furthermore, the separation of the two groups, deciduous and coniferous forests, was already at a very high level and exceeded the results from previous S1-based studies [14,15]. The increased accuracy could be related to the higher number of samples and/or the significantly smaller study area compared to the aforementioned studies. The larger number of derived S1 features could also play a role here, albeit comparisons across datasets are generally to be taken with caution.

The fact that the species within the conifer group were separated with satisfactory accuracies using microwave data—but not the deciduous species—is possibly related to the more distinct canopy surface roughness between conifers (as compared to the smoother deciduous species canopy surface). These results are in line with previous studies on tree species classification from S1 with slightly higher overall accuracies but fewer species (

Table 9. Summary of previous studies using Sentinel-1 (S1), Sentinel-2 (S2), and both combined for tree species classification and their achieved overall accuracies (OAs).

Satellite	No. of Species	Species Names	OA	Reference
S1	3	Quercus spp., Fagus sylvatica, Picea abies	72%	[16]
S1	4	Quercus robur, Betula spp., Picea abies, Pinus sylvestris	66%	[17]
S1	12	Fagus sylvatica, Alnus glutinosa, Fraxinus excelsior, Quercus spp., Prunus spp., Carpinus betulus, Acer spp., Picea abies, Pinus nigra, Pinus sylvestris, Larix decidua, Pseudotsuga menziesii	58%	Table 6
S2	4	Fagus sylvatica, Quercus spp., other broadleaf trees, coniferous trees	88%	[36]
S2	4	Sabina przewalskii, Picea crassifolia, Betula spp., Populus spp.	90%	[37]
S2	5	Larix spp., Pinus spp., Pinus mugo, Abies alba/Picea abies, broadleaf trees	84%	[38]
S2	5	Picea abies, Pinus silvestris, Larix × marschlinsii, Betula sp., Quercus robur	88%	[39]
S2	7	Picea sp., Pinus sp., Larix sp., Abies sp., Fagus sp., Quercus sp., other broadleaf trees	66%	[3]
S2	7	Acacia mearnsii, Eucalyptus dunnii, Eucalyptus grandiis, Eucalyptus mix, Pinus tecunumanii, Pinus elliotii, Pinus taedea	84%	[40]
S2	8	Fagus sylvatica, Quercus spp., Alnus spp., Betula pendula, Picea abies, Pinus sylvestris, Abies alba, Larix decidua	82%	[8]
S2	9	Fagus sylvatica, Betula pendula, Carpinus betulus, Abies alba, Acer pseudoplatanus, Larix decidua, European larch, Alnus incana, Pinus sylvestris, Picea abies	92%	[41]
S2	11	Alnus spp., Acer pseudoplatanus, Fagus sylvatica, Betula pendula, Carpinus betulus, Quercus spp., Picea abies, Pinus sylvestris, Larix decidua, Pseudotsuga menziesii	87%	[42]
S2	12	Fagus sylvatica, Alnus glutinosa, Fraxinus excelsior, Quercus spp., Prunus spp., Carpinus betulus, Acer spp., Picea abies, Pinus nigra, Pinus sylvestris, Larix decidua, Pseudotsuga menziesii	83%	Table 7
S2	12	Fagus sylvatica, Alnus glutinosa, Fraxinus excelsior, Quercus spp., Prunus spp., Carpinus betulus, Acer spp., Picea abies, Pinus nigra, Pinus sylvestris, Larix decidua, Pseudotsuga menziesii	90%	[6]
S2	12	Betula pendula, Quercus robur/pubescens/petraea, Quercus rubra, Populus spp., Fraxinus excelsior, Robinia pseudoacacia, Salix spp., Eucalyptus spp., Pinus nigra subsp. laricio, Pinus pinaster, Pinus nigra, Abies alba, Pseudotsuga menziesii, Cupressus spp.	0.90 *	[43]
S2	17	Fagus sylvatica, Alnus spp., Quercus petraea/robur, Quercus rubra, Betula pendula, Robinia pseudoacacia, Tilia cordata, Acer pseudoplatanus, Fraxinus excelsior, Populus spp., Carpinus betulus, Picea abies, Larix spp., Pseudotsuga menziesii, Pinus sylvestris, Pinus strobus, Pinus nigra	96% **	[44]
S1 + LS	4	Shorea siamensis, Shorea obtuse, Dipterocarpus tuberculatus, semi-evergreen/evergreen	79%	[18]
S1 + S2	6	Fagus sylvatica, Quercus spp., other broadleaves, Picea sp., Pinus sp., other conifers	63%	[7]
S1 + S2	6	Quercus mongolia, Betula spp., Populus spp., Armeniaca sibirica Larix spp., Pinus tabulaeformis	78%	[45]
S1 + S2	7	Acacia mearnsii, Eucalyptus dunnii, Eucalyptus grandiis, Eucalyptus mix, Pinus tecunumanii, Pinus elliotii, Pinus taedea	88%	[40]
S1 + S2	12	Fagus sylvatica, Alnus glutinosa, Fraxinus excelsior, Quercus spp., Prunus spp., Carpinus betulus, Acer spp., Picea abies, Pinus nigra, Pinus sylvestris, Larix decidua, Pseudotsuga menziesii	84%	Table 8

* mean F1-score instead of OA; ** OA influenced by high dominance of one class.

).

Test case 2, which was only based on S2 data, delivered significantly better results than Test case 1. This was expected as optical data reflect both structural and biochemical forest traits [46] and their temporal evolution, and S2 data are known to have a very high SNR and a good temporal coverage [47]. The differentiation of the two strata was again very good, but more importantly, now all species (deciduous and conifers) were separable. Separating the individual tree species yielded a satisfactory result with an OA of 83.2%, kappa of 0.806 and an improvement of 27.5 percentage points compared to the sole use of S1 (Test case 1). The result of Test case 2 could not reach the high accuracy of 88.7%, which was reached by using the original sample dataset and S2 scenes from several years [6]. However, the samples were increased in size and more balanced. Compared to other studies presented in Table 9, the OA of Test case 2 is within the range usually achieved with multitemporal S2 data.

The result of Test case 2 could be improved only marginally, 0.5 percentage points, by using additional S1 data (Test case 3). Including S1 data, however, we observed a shift in contributing features. Indeed, the MDA analysis revealed that Test case 3 included two very high ranking S1 parameters as input variables, which, therefore, had to replace at least two optical features from Test case 2. The high number of highly correlated features makes it very difficult to fully understand the MDA findings. After the MDA-Feature selection, band 9 never remained in the dataset. Therefore, the added value of band 9 in the discrimination of tree species depicted by Nkosi et al. [21] could not be confirmed.

For the study area, Wienerwald, and the employed RF classifier, it can be stated that if there are a sufficient number of S2 scenes available, no more added value is generated by using additional S1 data. We did, however, not evaluate the impact of the S2 orbit overlap on the classification accuracy, which certainly would negatively impact the obtained accuracies. On the other hand, temporal metrics (both parametric and using harmonics) in this study were only employed for S1 data.

The best monotemporal S2 result was achieved by the scenes from May and July, often a period with only few cloud-free data in Central Europe. Whenever data from several dates are available, the classification performance can be improved. Both variants of seasonal data (i.e., combining the three best and the three worst S2 scenes per season) demonstrated the added-value of multitemporal data.

Interestingly, in (optical) data-poor regions, the low S1 performance can be boosted significantly, if at least one S2 scene can be added as this adds valuable biochemical trait information. In this case, the OA of the classification can be increased by ca. 5.0 percentage points on average. This finding is significant in that S1 scenes, unlike S2 scenes, are always available due to the nature of their sensor’s active microwaves, but are only marginally performant when used alone. If, however, several S2 scenes are available, the added value of S1 is increasingly equalized and decreases to 1.7 percentage points with a composite of MAS-S2 scenes and to 1.5 percentage points with a composite of LAS-S2 scenes. This is in line with the findings of Mngadi et al. [40] and Waser et al. [15]. Even using seasonal S2-composites as in other studies [48], overall accuracies of the full multitemporal datasets as in Test case 2 and Test case 3 were far from achieved.

5. Conclusions

In this study, the performance of the Sentinel-1 and Sentinel-2 satellite pairs, both individually and in various combinations, is presented. Twelve tree species, seven deciduous and five coniferous, of the Austrian Biosphere Reserve Wienerwald were classified using Breiman’s random forest. While the results using only Sentinel-1 data were not satisfactory, the ability of the Sentinel-2 satellites to classify tree species was demonstrated once again. The greatest increase in accuracy can be achieved by using multitemporal Sentinel-2 data. In areas with insufficient coverage of optical satellites, Sentinel-1 can add value to classification accuracy. Seasonal Sentinel-2 composites have advantages over monotemporal classifications, but preference should be given to a full time series whenever possible. If sufficient Sentinel-2 data are available, the added value of Sentinel-1 data is only marginal, so that the effort to acquire data is offset by the added value of increased accuracy. However, for large-scale applications, the possibilities of acquiring cloud-free Sentinel-2 time series are often the limiting factor. In such cases, the advantage of the Sentinel-1 time series is obvious. The next steps would be further combinations with other datasets such as LiDAR or hyperspectral data (e.g., EnMAP). For a better understanding of the results and the relationship between vegetation structure and reflectance properties, radiative transfer models (RTMs) should be consulted.