A Neural Network Method for Classification of Sunlit and Shaded Components of Wheat Canopies in the Field Using High-Resolution Hyperspectral Imagery

Abstract

(1) Background: Information rich hyperspectral sensing, together with robust image analysis, is providing new research pathways in plant phenotyping. This combination facilitates the acquisition of spectral signatures of individual plant organs as well as providing detailed information about the physiological status of plants. Despite the advances in hyperspectral technology in field-based plant phenotyping, little is known about the characteristic spectral signatures of shaded and sunlit components in wheat canopies. Non-imaging hyperspectral sensors cannot provide spatial information; thus, they are not able to distinguish the spectral reflectance differences between canopy components. On the other hand, the rapid development of high-resolution imaging spectroscopy sensors opens new opportunities to investigate the reflectance spectra of individual plant organs which lead to the understanding of canopy biophysical and chemical characteristics. (2) Method: This study reports the development of a computer vision pipeline to analyze ground-acquired imaging spectrometry with high spatial and spectral resolutions for plant phenotyping. The work focuses on the critical steps in the image analysis pipeline from pre-processing to the classification of hyperspectral images. In this paper, two convolutional neural networks (CNN) are employed to automatically map wheat canopy components in shaded and sunlit regions and to determine their specific spectral signatures. The first method uses pixel vectors of the full spectral features as inputs to the CNN model and the second method integrates the dimension reduction technique known as linear discriminate analysis (LDA) along with the CNN to increase the feature discrimination and improves computational efficiency. (3) Results: The proposed technique alleviates the limitations and lack of separability inherent in existing pre-defined hyperspectral classification methods. It optimizes the use of hyperspectral imaging and ensures that the data provide information about the spectral characteristics of the targeted plant organs, rather than the background. We demonstrated that high-resolution hyperspectral imagery along with the proposed CNN model can be powerful tools for characterizing sunlit and shaded components of wheat canopies in the field. The presented method will provide significant advances in the determination and relevance of spectral properties of shaded and sunlit canopy components under natural light conditions.

1. Introduction

Hyperspectral imaging (HSI) was a breakthrough for remote sensing applications [1,2,3,4]. It combines imaging and spectroscopy to attain simultaneously and non-invasively both spatial and spectral information and forms a three-dimensional data cube. HSI provides a vast source of information by sampling the reflective portion of the electromagnetic spectrum covering a wide range from the visible region to the short-wave infrared region. These optical datasets, known as hypercubes, comprise two spatial dimensions and one spectral dimension. Each plane of the hypercube is a grayscale image corresponding to a single wavelength with each pixel displaying the radiance intensity reflected by the observation. Thus, each pixel of the hypercube contains the spectral signature of the underlying object. Since the spatial information is available, the source of each spectrum can be located, which makes it possible to investigate the light interactions with the material surface, vegetation, plant component, etc.

In the past decades, a surge of interest in hyperspectral imaging has been seen in the life sciences with applications in fields as diverse as food quality and control [5], pharmaceuticals [6], healthcare [7] and agriculture [1,8,9]. It has been applied in precision agriculture for weed detection [10], plant disease and stress detection [2], plant water and nitrogen content [11]. Hyperspectral technologies are becoming one of the most promising techniques to assess functional plant traits [12] in plant phenotyping [8,9]. Despite the advantages of using HSI in these areas of research, hyperspectral imaging still faces various tradeoffs and is not exempt from issues and drawbacks. For instance, spectral reflectance captured by HSI at the canopy scale is more complex and influenced by multiple variabilities, such as factors associated with plant architecture and geometry, soil background, and leaf scattering properties [13,14]. The acquisition of HSI in uncontrolled environments produces additional challenges including rapidly varying light exposure and the influence of wind turbulence [15], especially for push-broom cameras. Also, the high dimensionality of spectral bands and the high spatial resolution pose serious challenges for the quantitative analysis of the data. Although the high-dimensional features may provide some advantages for more accurate classification, it may cause algorithmic instability, i.e. the Hughes phenomenon [16] which causes a negative impact on the accuracy and efficiency of data analysis models.

Computer vision is a fundamental yet important step in extracting quantitative and qualitative information from hyperspectral imaging. The image analysis techniques range from straightforward extraction of average spectra from entire images (which is equivalent to the use of non-imaging spectrometers) to segmenting plants of interest. One aim of segmentation is to eliminate complex and varying backgrounds (e.g., soil, rock, etc.) within the sensor field of view that do not correspond to the object(s) of interest and retain spatial information about the patterns of spectral variation. HSI segmentation methods can be grouped into three main categories: threshold-based methods, model-based methods and feature-based methods. In threshold-based techniques, a fixed threshold is applied to a selected vegetation index, such as normalized difference vegetation index (NDVI) [17] or Photochemical Reflectance Index (PRI) [18], to segment residual background or target plants. These methods are very sensitive to the choice of threshold, which tends to be a subjective choice or requires a trial-and-error approach. Model-based methods, on the other hand, use prior information such as geometric features of objects and digital surface model data to reconstruct 3D models. Due to the complexity associated with 3D models, model-based methods are computationally slow and are difficult to implement in practical applications [18,19]. Moreover, 3D models are often not very accurate due to misalignment between 3D data and hyperspectral images as they are obtained from different resources and geo-referenced independently. The feature-based methods mainly consist of spectral classification and spatial classification. Spectral classification methods rely on the spectral signature of each pixel in hyperspectral images, whereas spatial classification techniques only employ the spatial information and are limited to one spectral band without fully exploiting the spectral information in HSI. The difficulty associated with these models involves algorithmic instability [20] due to the high dimensionality of using an entire spectral signature.

In the case of vegetation mapping, the identification of shaded and sunlit (non-shadowed) regions has also become an important part of remote sensing. Vegetation canopies represent complex dynamic spaces in which light is absorbed or transmitted by leaves; thus, shadows might have a considerable impact on the prediction of biochemical or physiological status [21,22]. Under natural conditions, shadows may occur due to the canopy structure known as self-shadows, or when a fraction of direct light from solar illumination is blocked by objects presented in the scene (e.g., the observing platform in our case) known as cast-shadows. Nonetheless, shadows may cause a reduction or total loss of information in a hyperspectral image, which leads to the corruption of biophysical parameters derived from pixel values, such as vegetation indexes [23,24].

Several studies investigated the impact of shadow on plant predictors and only focused on forestry applications with a predominance on the effect of the shaded section (self-shadows) on the PRI for biomass, light use efficiency and photosynthesis [25,26] and orchard to only works on pure vegetation pixels [27,28,29]. Camino et al. [30] investigated the effect using pure vegetation pixels or mixed pixels from tree crowns using HSI and thermal imaging to estimate water stress indicators. Maimaitiyiming et al. [31] developed a weighted index to consider the respective weights of sunlit and shaded pixels on the calculation of the sun-induced fluorescence index and the crop water stress index. However, to date, little is done to cereal crops to study the impact of shadow on vegetation indices and their importance as predictors of leaf area index, vegetative biomass, chlorophyll and nitrogen content and grain yield. Most of the studies are based on non-imaging spectrometers which cannot provide spatial information and do not allow investigation of the effect of shade on the aforementioned yield components. However, the rapid development of high-resolution imaging spectroscopy sensors mounted on close-range platforms opens new opportunities to investigate the effect of shade on individual plant organs, such as leaves and reproductive organs. Yang et al. [32] investigated the effect of shaded areas on PRI to evaluate water stress status in winter wheat. In rice, Zhou et al. [33] investigated the effect of shade on chlorophyll content prediction. A threshold-based method was proposed to segment panicles from vegetation in both sunlit and shaded areas.

Spectral-based classification techniques utilize spectral features to identify objects in HSI. These techniques can be grouped into spectral matching and statistical characteristics categories. In spectral matching methods, the discrimination of targets such as plant variety is based on the comparison of the similarity of the given spectrum with the reference spectrum. Based on the fine spectral information of HSI, methods such as spectral information divergence (SID) [34] and spectral angle mapper (SAM) [35] do not require complex analysis and dimension reduction; however, they highly rely on reference spectral data. On the other hand, the classification methods based on statistical characteristics, such as support vector machines (SVM), and logistic regression classifiers offer state-of-the-art performances in hyperspectral image classification [36,37] by finding the optimal decision boundaries among different classes. However, the effectiveness of such classifiers depends on the selection of some critical hyperparameters to control the learning process, which defines the classification model. There is a growing interest in applying convolutional neural networks (CNNs) as promising tools for computer vision analysis. CNNs endorse the powerful visual analysis capabilities that outperformed traditional computer vision techniques [38,39]; however, the complexity of HSI data structure and lack of available training samples pose a challenge in employing CNNs models for HSI. Recently, CNNs techniques have made breakthroughs in processing hyperspectral images [40,41,42]. Some models used the spectral domain directly, whereas other methods took the spatial features of hyperspectral images into account [43]. Nonetheless, both techniques, have shown promising performance in classifying hyperspectral imagery.

In this study, we introduce an image analysis pipeline along with a 1D-CNN model to automatically characterize the spectral variation of wheat leaves and spikes in shadowed and non-shadowed regions. To fulfil these objectives, two CNN models are presented for feature extraction and spectral-based HSI classification of shaded and sunlit components at canopy level. In the first method, we incorporate pixel vectors of the full spectral features as inputs to the CNN model. The second model integrates the dimension reduction technique known as linear discriminate analysis (LDA) along with CNN to increase the feature discrimination and improves computational efficiency. Finally, the performance of the CNN-based techniques is evaluated from different perspectives such as classification accuracy and computational time.

2. Materials and Methods

In this study, several image analysis steps are developed to extract relevant information from the HSI data. The spectral properties are extracted at different growth stages to evaluate spectral variations of shaded and sunlit canopy components. The entire pre-processing and image analysis steps are coded in python environment using PyTorch [44] and OpenCV [45] libraries. Thus, it creates an end-to-end open-source pipeline for analysis of HSI data of multiple crop cultivars at the plot level.

2.1. Imaging Setup and Data Acquisition

The hyperspectral images at canopy level were collected using a Hyperspec^® Inspector™ VNIR camera (Headwall Photonic). The VNIR camera is a push-broom imaging system that collects reflected light through an imaging slit. One row of spatial pixels is collected per frame as motion occurs, with each pixel containing the spectral data [46]. The target is scanned line by line, and spatial images are formed by recording simultaneously the spectral information of pixels distributed in a scan line (across-track direction), while the mirrors move horizontally. The sensor collects data within the 400 to 1000 nm region of the electromagnetic spectrum with a 0.7 nm step and an FWHM (full width at half maximum) image slit of 2.5 nm. It results in a hyperspectral data cube (hypercube) of 925 spectral bands with a dynamic range of 16 bits. The hyperspectral images were collected under natural light conditions using the fully automated Lemnatec Field Scanalyzer platform (

Table 1. Summary of image acquisition conditions for the data collected in this study.

Date	Growing Stage	Time (GMT + 1)	$$\begin{gathered} \mathbf{Range} \mathbf{of} \mathbf{PAR} \mathbf{Values} \\ \left(\mathsf{\mu }\mathbf{m}\mathbf{o}\mathbf{l}/\mathbf{m}^2\mathbf{s}\right) \end{gathered}$$	Spatial and Spectral Resolution
21 May 2019	GS39: Flag leaf fully emerged	9:30–12:30	1204–1856	533 × 667 × 925
6 June 2019	GS57: GS59-advanced heading time	16:18–17:33	975–1429	1600 × 1846 × 925
19 June 2019	7 days after anthesis	17:44–19:28	230–472	1600 × 1846 × 925
4 July 2019	22 days after anthesis	9:36–11:36	1267–1645	533 × 667 × 925

). The data were collected at different growth stages according to the scale used by the AHDB wheat growth guide (AHDB Wheat Growth Guide, 2018 [47]): flag leaves fully emerged (GS39), advance heading time (GS57/59), 7 days after anthesis, and 22 days after anthesis (which is corresponding to early/mid stage of the senescence period). The VNIR camera (Figure 1A) was set up with two spatial configurations. The first configuration acquired images at a resolution of 533 × 667 pixels with 925 bands, and a second configuration collected images from the VNIR sensor at a resolution of 1600 × 1846 pixels with 925 bands. In both settings, the exposure time was fixed manually to adapt to brightness variations between scans.

The experiment was conducted in 2018–2019 at the Field Scanalyzer platform [48] located at Rothamsted Research, UK (51°48′34.56′′N, 0°21′22.68′′W). On the 25 October 2018, four wheat commercial cultivars (Triticum aestivum L. cv. Crusoe, Hereward, Istabraq and Maris Widgeon) were sown in three blocks according to a split plot design for a total of 72 plots of 3 m $\times$ 1 m with a planting density of 350 seeds/m². Nitrogen (N) treatments were applied as ammonium nitrate in two splits during the spring for a total rate of 50 kgN.ha⁻¹ (N1), 100 kgN.ha⁻¹ (N2), 150 kgN.ha⁻¹ (N3), 200 kgN.ha⁻¹ (N4), 275 kgN.ha⁻¹ (N5) and 350 kgN.ha⁻¹ (N6). A first application split of 50 kg/ha was done on the 8 March 2019 and the remaining nitrogen was applied on the 10 April 2019. The experiment was managed according to local agronomic practices.

2.2. Pre-Processing of Raw Hyperspectral Images

Prior to any analysis, pre-processing the raw hypercube is mandatory to reduce artefacts generated during measurement and to normalized spectra from ambient illumination. In this study, the pre-processing steps involve (i) removing the effect of the illumination system, (ii) down-sampling by integrating/averaging hypercube images with similar wavelength, (iii) reducing noise:

Reflectance factors: Imaging under natural light conditions results in an additional challenge of rapidly varying light exposure. Thus, data radiometric correction is required to eliminate the spectral non-uniformity of the illumination and the influence of the dark current. In our experiment, the calibration was performed based on the flat field calibration method using a white reference panel (Zenith Lite™ Ultralight Targets 95%R, Sphereoptics^®) mounted on a tripod (Figure 1B). The reference panel was scanned after every seven plots (~15 mins interval). Dark reference images were collected during the night without any light source. Then, for each, the reflectance (R) of each wavelength (w) was calculated using the closest reference scan in terms of time and/or light intensity (Photosynthetic Active Radiation collected simultaneously to the scan) following the Equation (1) [49,50].

$$R^w=\frac{{\mathit{DN}}^w{}_{\mathit{raw\_sample}}-{\mathit{DN}}^w{}_{\mathit{DR}}}{{\mathit{DN}}^w{}_{\mathit{WR}}-{\mathit{DN}}^w{}_{\mathit{DR}}} \times Target coefficient$$

(1)

where $D{N}_{raw\_sample}$ is the digital number of a pixel sample; $D{N}_{DR}$ and $D{N}_{WR}$ are the average values of the dark and white reference image at the same wavelength (w) as the sample image.

The target coefficients are correction factors associated to each wavelength. The values are obtained after calibration using an integrative sphere by the manufacturer.

Down-sampling: in order to match the spectral resolution of the reference target, spectra were down-sampled with an averaging window with a spectral width of 1 nm. As a result, the spectral resolution was reduced to 600 bands. This also reduces the computational complexity and the instrumentation noise from the spectrometer.

Smoothing of spectral data: To correct baseline drifts in NIR spectra and aid denoising, a method such as low-pass filter is frequently employed in HSI. Denoising aims to eliminate spikes and to smooth the spectral curves of each pixel while retaining the variations across different wavelengths. In the low-pass filter, noisy spectral values are replaced by taking the local average of neighboring data points. Since nearby spectral values measure very nearly the same underlying value, averaging can reduce the level of noise with minimum bias. The simplest low-pass filter computes a moving average of a fixed number of spectral data. However, the moving average filter is particularly aggressive and damaging when the filter passes through peaks that are narrower than the filter width. In this work, we used the Savitzky–Golay filter [51], an exceptionally effective and computationally fast smoothing filter. The Savitzky–Golay filter performs a least square fit of a small set of consecutive spectral data to a polynomial and takes the calculated central point of the fitted polynomial curve as the new smoothed spectral data point. It should be noted that too small a window will lead to the introduction of large artifacts in the corrected spectra and to a reduced signal-to-noise ratio. On the other hand, the larger the size of the window, the smaller the distinction between full and moving window pre-processing [52]. In our case, the filter width w = 11 and polynomial degree d = 2 were the optimal values for this study.

2.3. Training Dataset and Dimensionality Reduction

Upon completion of the pre-processing step, a supervised model is developed to classify crop canopy components into five classes of shaded leaves (SHL), shaded ears (SHE), sunlit leaves (SL), sunlit ears (SE), and background (BG). To build a supervised classification model, we first obtained a manually annotated dataset from shaded and sunlit wheat canopy components. Due to the sufficiently high spatial resolution of the hyperspectral images of the VNIR camera, we were able to build the spectral libraries of leaves and ears in both shaded, and sunlit areas derived from the user-defined region of interests (ROIs). ROIs were defined manually as rectangular areas based on visual identification in the HSI “pseudo” RGB data composed of the three bands 620 nm (Red), 535 nm (Green), and 445 nm (Blue). The annotated data was collected from 23 hyperspectral images of wheat cultivars at different crop growth stages. The total number of annotated data for 5 classes of SL, SE, SHL, SHE, and BG were 119,447, 164,223, 11,644, 4361, and 227,232 pixels, respectively. Each annotated patch is represented by $w\times h\times \lambda$; where w × h is the width and height of the rectangle and $\lambda$ is the number of wavelengths ($\lambda$ = 600). It should be noted that cast shadows did not occur during the data collection on 6th and 19th June; thus, the shadow samples (SHL and SHE) derived only from the collected data on 21st May and 4th July.

The high resolution of hyperspectral imagery employed in this study poses a significant challenge to the quantitative analysis of the hypercube. It can significantly increase the computational burden and storage space, leading to an increase in the data processing. As shown in similar studies [53], it is desirable to apply a spectral redundancy to select the most characterizing compact feature set. First, the hypercube ($x\times y\times \lambda$) is rearranged into a 2-D spectral matrix of dimension $N\times \lambda$, where $N=x\times y$ is the total number of pixels; thus, each row represents the reflectance values from all bands at one pixel. Such a vectorization process will allow us to employ the second-order analysis procedures for reducing the data complexity. We applied a linear discriminant analysis (LDA) to find a linear combination of features that characterizes our pre-selected classes (SHE, SHL, SL, SE, BG). As opposed to principal component analysis (PCA), which takes into account only the spectral data and its variance regardless of their grouping, LDA explicitly attempts to make use of the labels to maximize the distance between five classes (SHL, SHE, SL, SE, BG). Figure 2, illustrates scatter plots of PCA and LDA to demonstrate the grouping, similarities and differences among classes. Figure 2A,B visualizes the data in two discriminant coordinates found by LDA and PCA. As shown in both figures, LDA performed better in separating the classes; on the other hand, the classes are not as clearly separated using the first two principal components found by PCA, even though together the first two principal components contain over 90% of the information.

2.4. The CNNs Framework for Spectral Classification

HSI classification is a fundamental yet important step to provide primary information for the subsequent tasks. The hierarchical architecture of CNNs can be an effective way to learn spectral signatures for HSI classification. In this study, we employed 1D-CNN technique to model the interclass appearance and shape variations of spectral channels obtained by the VNIR camera to improve the power of accurately differentiating wheat canopy components. The proposed CNN technique extracts effective features from the spectrum with the help of class-specific information, which is provided by the training samples.

The CNN framework was constructed by stacking several convolutional layers and max pooling layers to form a deep architecture (Figure 3). Two CNN models were tested for characterizing spectral signatures. In the first method (hereafter referred to as CNN-RAW), the full spectral bands ($\lambda =600$) were selected as the input to the network, whereas the second method (hereafter referred to as CNN-LDA) integrated the linear discriminate analysis (LDA) along with the 1D-CNN to increase the feature discrimination and improve computational efficiency. Both networks contain two convolutional layers, two pooling layers, fully connected layer and the output layer which is the label of the pixel vector assigned to each canopy component (SHL, SHE, SL, SE, and BG). Also, to address the problem of overfitting, regularization strategy was added to the network, including Rectified Linear Units (ReLU) and dropouts to achieve better model generalization.

$$x_i^l=b_i^l+{{\displaystyle \sum }}_{k=1}^{N_{l-1}}conv1D\left(w_{ki}^{l-1}, s_k^{l-1}\right)$$

(2)

where $x_i^l$ represents the input, $b_i^l$ denotes the bias of the $i^{th}$ neuron at layer l, $s_k^{l-1}$ is the output of the $k^{th}$ neuron at layer l − 1, $w_{ki}^{l-1}$ represents the kernel from the $k^{th}$ neuron at layer l − 1 to the $i^{th}$ neuron at layer l. $conv1D\left(.,.\right)$ is used to perform 1D convolution without zero-padding.

Figure 3 illustrates the proposed 1D-CNN framework. ReLU is selected as the activation layer to increase the non-linear representations in the network. After three convolutional layers, ReLU layers and dropouts, the input pixel vector is converted into a feature vector. Then, the fully connected layer merges features obtained in the previous layer. Finally, logistic regression classifier is used to fulfil the classification step. The logistic regression classifier uses softmax as its output-layer activation to ensure that the activation of each output unit sums to 1 so that the output can be deemed as a set of conditional probabilities. The logistic loss function is selected to calculate the error between the predicted label and the ground-truth. The goal is to minimize the logistic loss function. We kept track of our losses after each epoch (set to 15), which represents the number of training iterations.

Figure 4 and Figure 5 present the corresponding classification maps to label canopy components using the proposed method (CNN-RAW). The classification result leads to a pseudo color map where each pixel vector of the spectral cube is assigned to a unique label. In Figure 4, shadowed vegetation (SH-All), sunlit vegetation (S-All), and background (BG) pixels were segmented and assigned to a unique label, whereas in Figure 5, the classification map is shown for three classes of sunlit wheat ears (SE), sunlit leaves (SL) and background (BG). Both figures illustrate the feasibility of classifying the HSI in field environments at the canopy scale with high accuracy. In the end, to capture the spectral properties of each canopy component, each assigned label is used as a binary mask over the hypercube to get an average reflectance value for each canopy component at all wavelengths (400–1000 nm).

3. Results

3.1. Classification Accuracy Assessment

To objectively assess the performance of HSI classification, we selected three widely used classification measurements: average accuracy (AA), F-score and recall-score (Equation (3)). Accordingly, the classification accuracy of the proposed methods was compared with conventional classification methods, such as stochastic gradient descent (SGD), and support vector machine (SVM) classifiers. It should be noted that the baseline parameters in scikit-learn API are used for both models.

To validate the effectiveness of the presented classifiers, we conducted stratified k-fold cross-validation method with 10 folds and three repeats.

Table 2. Classification accuracy assessment.

	RAW			LDA
	Acc.	Fscore	Recall	Acc.	Fscore	Recall
CNN	$0.986$	0.979	0.983	$0.973$	0.953	0.945
SVM	$0.980$	0.972	0.972	$0.974$	0.957	0.956
SGD	$0.981$	0.974	0.973	$0.964$	0.950	0.950

summarizes the AA, F-score, and recall-score for each model, whereas Figure 6 depicts a box and whisker plot to summarize the distribution of accuracy scores. As shown in both Table 2 and Figure 6, the CNN model outperformed the SVM and SGD classifiers while using RAW spectral features with an average accuracy of 98.6%. On the other hand, SVM outperformed the other two methods using LDA with an average accuracy of 97.4%. The CNN and SGD classifiers came second and third with an average accuracy of 97.3% and 96.4%, respectively. Overall, the classifiers with the full spectral (RAW) achieved higher HSI classification accuracy than the dimensionality reduction LDA.

$$Accuracy=\frac{\mathrm{TP}+\mathrm{TN}}{\mathrm{TP}+\mathrm{TN}+\mathrm{FP}+\mathrm{FN}}; Precision=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}};$$

$$Recall=\frac{\mathrm{TP}}{\mathrm{TP} + \mathrm{FN}};F_{score}=2\frac{P\times R}{P+R}$$

(3)

where TP: true positive; TN: true negative; FP: false positive; FN: false negative.

3.2. Computational Cost

In this section, the computational cost of the proposed methods is presented. As discussed in Section 3.1, the CNN-RAW method achieved a higher HSI classification accuracy; however, it is computationally expensive since it uses the full spectral features. The computational complexity of the CNN-RAW classification is $O\left(\lambda N\right)$, where $\lambda$ is the number of wavelengths and N is the total number of pixels. On the other hand, the computational cost of the CNN-LDA method is $O\left(mN\right)$, where m is the number of LDA (m = 2) which is considerably lower than computing the full wavelength range of 400–1000 nm. The CNN-LDA method is also computationally faster than the CNN-RAW model. For this test, the average computational time was calculated over processing several HSI images with spatial resolutions of 553 $\times$ 667 with 600 bands. All the tests were performed on a PC with 3.2 GHz Quad-Core Intel Core i5, 16 GB memory, using Macintosh OS 11. As expected, the introduction of LDA dimensionality reduction method greatly improves the computational time. The CNN-LDA method is nearly two times faster on average (5.9 s) than the CNN-RAW algorithm with 11.12 s.

3.3. Comparison of the Reflectance Amplitude and Absorption Feature between Shaded and Sunlit Canopy Components

In this section, we present the spectral signature of five classes extracted from the CNN-RAW method. The classes include sunlit leaf and ears (SL and SE), shaded leaves and ears (SHL and SHE), and background (BG, Figure 7A). As the reflectance spectra from the shaded organs display low amplitudes, a normalization method known as continuum removal is applied to compare the shape of absorption features (Figure 7B). This approach allows comparison of individual absorption features from a common baseline by enhancing differences in absorption strength while normalizing for absolute differences of the reflectance peaks in SHL, SHE, SL, SE, and BG.

The reflectance signature of the sunlit fractions, SL and SE, shows higher reflectance values across the 400–1000 nm range compared to their shaded counterparts (SHE and SHL), which can be attributed to the lower irradiance of shadowed classes and thus a lower reflected signal (Figure 7A). While the amplitude and the reflectance values of the shaded organs are lower than the sunlit counterparts, all classes display a similar spectral signature pattern with lower reflectance in the visible domain (with a small peak in the green region), due to the light absorbed by chlorophyll, and higher reflectance in the NIR domain. This is confirmed in Figure 7B by illustrating the continuum removal reflectance values (CRR) which shows a drop in intensity at ~490 nm in all four classes, a peak around ~540 nm, followed by a continuous drop at 680 nm, before increasing sharply in the red-edge to reach its maximum in the NIR region. Interestingly, the reflectance of sunlit ears and leaf displays the highest intensities in the 510 to 570 nm range, but this is not the case for the sunlit leaves with the CRR. Indeed, ears obtained the highest intensities in the 510–570 nm range in both sunlit and shaded conditions. This may be attributed to the CRR as it is maximising the contrasting features of the reflectance spectra by normalising them to a common baseline. In our case, it would mean the relative difference between the peak in the green region compared to the baseline is higher for shaded ears than for the sunlit leaves.

3.4. Effects of Shadows in Vegetation Indices

The output of the CNN-RAW has been used to compute a set of vegetation indices for each of five classes: background, sunlit leaves and ears, shaded leaves and ears (

Table 3. List of vegetation indices used in this study.

Vegetation Index	Calculation Formula
Difference Vegetation Index (DVI) [55]	$R_{800}-R_{670}$
Enhanced Vegetation Index (EVI) [56]	$2.5\times \left[\left(R_{800}-R_{680})/(R_{800}+6\times R_{680}-7.5\times R_{450}+1\right)\right]$
Greenness Index (G) [57]	$R_{554}/R_{677}$
Improved SAVI with self-adjustment factor L (MSAVI) [58]	$0.5\times \left\{2\times R_{800}+1-\sqrt{{\left(2\times R_{800}+1\right)}^2-8\times \left(R_{800}-R_{670}\right)}\right\}$
Modified Simple Ratio (MSR) [59]	$\left(R_{800}/R_{670}\right)-1/\sqrt{R_{800}/R_{670}+1}$
Modified Triangular Vegetation Index (MTVI) [60]	$1.2\times \left[1.2\times \left(R_{800}-R_{550}\right)-2.5\times \left(R_{670}-R_{550}\right)\right]$
Normalized Difference Vegetation Index (NDVI) [61]	$\left(R_{800}-R_{680}\right)/\left(R_{800}+R_{680}\right)$
Optimized Soil Adjusted Vegetation Index (OSAVI) [62]	$\left(1+0.16\right)\left(R_{800}+R_{670}\right)/\left(R_{800}+R_{670}+0.61\right)$
Photochemical Reflectance Index (PRI) [63]	$\left(R_{531}-R_{570}\right)/\left(R_{531}+R_{570}\right)$
Soil-adjusted Atmospherically Resistant Vegetation Index (SARVI) [64]	$\frac{R_{800}-\left(R_{670}- 1 \times \left(R_{445} - R_{670}\right)\right)}{\left[R_{800}+\left(R_{670}-1\times \left(R_{445}-R_{670}\right)\right)+0.5\right]\times \left(1+0.5\right)}$
Triangular Veg Index (TVI) [65]	$0.5\times \left[120\times \left(R_{750}-R_{550}\right)-200\times \left(R_{670}-R_{550}\right)\right]$
Vegetation Stress Ratio (VS) [66]	$R_{725}/R_{702}$

, Figure 8). The aim here was to investigate the sensitivity of different VI in the canopy components and how they are affected by the shadows. The selection of VIs was inspired by the works in [33,54]. Figure 8A shows the pixel distribution of the sunlit and shaded fraction regardless of the organ component. It clearly shows that indices like DVI, EVI, MSAVI, MTVI, OSAVI, SARVI and TVI are able to discriminate background from vegetation as well as the sunlit fraction from the shaded fraction. All these indices are displaying a similar pattern in pixel distribution with low values for the background, intermediate values for the shaded fraction and the highest values for the sunlit fraction. Conversely, NDVI, G, MSR, PRI and VS display higher values for shaded areas compared to the sunlit areas, with no clear distinction of the pixel distribution able to discriminate both.

Figure 8B,C show the pixel distribution of the VIs for the organs, leaves and ears, for shaded and sunlit conditions, respectively. Only four indices are able to discriminate between ears and leaves within the shaded area: MSR, NDVI, PRI and VS (Figure 8B). For the sunlit area, MSAVI, MSR, NDVI, OSAVI, PRI, SARVI and VS show a potential to discriminate the ears from the leaves. However, most of them show an overlap between the pixel distribution of the two areas. Only the PRI seems to clearly distinguish the ears from the leaves in sunlit conditions.

4. Discussion

This study aimed to produce an accurate classification of wheat canopy components in shadowed and non-shadowed areas using high-resolution hyperspectral images. This was achieved by developing CNN-based models to automatically learn and evaluate the spectral characteristics of shaded and sunlit wheat canopy components. The analysis was carried out on high-resolution hyperspectral images acquired from the ground-based phenotyping platform, known as the Field Scanalyzer [48].

Two CNN based techniques were presented. In the first technique (CNN-RAW), pixels from the full spectral bands are used as the input layer. In the second method (CNN-LDA), the dimensionality of the spectral data was first reduced using the LDA technique and then the CNN model was deployed. The performance of the CNN techniques has been validated from the aspect of classification metrics and computational time. The accuracy of the proposed models was also compared against conventional classification methods (Section 3.1). As shown in Table 2, the proposed methods achieved over 98% (CNN-RAW) and 97% accuracy (CNN-LDA). Although less classification accuracy was achieved in CNN-LDA, it enhances the interpretability of the spectra information by replacing the original variables with a group of new variables while preserving original information. CNN-LDA also reduced the computational complexity and reduced the processing time by half compared to the CNN-RAW method, as described in detail in Section 3.2.

Previous studies on HSI classification often focused on the non-shadow portions of the canopy and neglected the importance of shaded regions [15,67,68]. This is partially due to the insufficient spatial resolution of HSI or poor signal to noise ratio in the shaded pixels. In this study, we demonstrated that high-resolution hyperspectral imagery can be used to characterize shaded and sunlit components in wheat canopies. The presented method can be used as a powerful tool to the reflectance signal across the spectrum from individual wheat canopy components. The results showed that shaded and sunlit components display separate spectral signatures and can be used to understand canopy biophysical and chemical characteristics. As shown in Figure 7A, the sunlit components (SE and SL) exhibited higher reflectance values than their shaded counterparts (SHL and SHE). In particular, the NIR reflectance of non-shadowed spikes and leaves was nearly three times bigger than the shaded counterparts. The absorption features were also investigated via continuum-removed reflectance.

Finally, it was shown that shade had significant effects on the estimation of vegetation parameters at the canopy scale. Some vegetation indices exhibited more distinctive distribution between shaded and sunlit components (Figure 8). For instance, DVI, EVI, MSAVI, MTVI, and TVI in SH-All vs S-All (Figure 8A), MSR and VS in SHL vs SHE (Figure 8B), and finally PRI and VS in SL vs SE (Figure 8C) showed separate distribution intervals between canopy components. This illustrates that the aforementioned VIs can play roles in discerning shaded and sunlit components as well. The results also show that VIs can be utilized to find key features that uniquely characterize shaded and sunlit spectroscopy pixels within canopies.

5. Conclusions

In this study, a one-dimensional convolutional neural network method was proposed to map shadowed and non-shadowed wheat canopy components from high-resolution hyperspectral imaging. The presented method has a unique capability to learn directly from the raw spectral signatures and voids the need for designing handcrafted feature extraction and it can surpass traditional classification approaches. Moreover, the 1D-CNN has a relatively shallow architecture with a smaller number of hidden layers and neurons compared to the state-of-the-art 2D-CNNs techniques. Thus, it is much easier to train and can offer the minimal computational complexity which is suitable for hand-held hyperspectral devices with limited computational power. However, the spectral-based 1D-CNN method does not take full advantage of the three-dimensional data cube characteristics. In future work, we aim to incorporate spatial information into the spectral-based feature extraction model to improve the classification performance of HSI in more complex scenes.

Furthermore, in terms of canopy component characterization, canopy physiological response to changing light conditions can cause additional complications to the assessment of component mapping. For instance, as discussed in another study [33], the spectral differences between shaded lower layer canopy components and sunlit upper layer counterparts likely depend not only on the illumination variations, but also on the non-uniform distribution of chlorophyll and nitrogen. Therefore, multi-angle viewing hyperspectral imaging is likely to be more effective than only vertically downward-facing sensors.