1. Introduction
The progression in remote-sensing technology has presented the world with invaluable sources of crop-related information. Through this technology, crops can be better quantified for their management and yield estimation [
1,
2]. However, discriminating small-scale crops requires remotely sensed images with very high spatial resolutions, typically in centimeters [
3,
4]. Unmanned aerial vehicle (UAV) systems, in particular, have recently emerged as a valuable source of high-spatial-resolution remote-sensing data, offering substantial benefits in relation to cost, adaptability, and precise spatial resolution. UAV systems fly at a low altitude, allowing acquisition of multispectral images with very high spatial resolution (VHSR) [
5]. With these advantages, UAV systems are increasingly developing to be an effective way to complement satellite remote sensing [
6,
7]. From a multispectral remote-sensing perspective, detection of features from imagery with an accurate and computationally efficient approach is one of the foremost modern challenges in image pattern recognition [
8]. From the crop perspective, pixels that make up an image contain unique information about the nature and type of crop at a particular location. As such, changes in pixel values across the image become the basis for defining the spatial pattern of a particular crop in that imagery. Moreover, analysis of the interaction of scene illumination with the reflectance and geometry of respective illuminated crop types may serve as the basis for quantifying the amount of the different types of crops.
Just like with any other remote-sensing end products, the quality of the UAV photogrammetric end products still requires serious attention if accurate spectral characterization of crops is to be achieved with these products [
9]. However, correction of these images is usually confined to noise reduction and detector errors [
10]. UAV data are subjected to radiometric quality issues as a result of variations in solar radiation, atmospheric effects, sensor viewing angle, and calibration mistakes [
11]. This underscores the need to remove these errors, with a view to produce illumination-consistent and atmospheric-independent imagery [
12]. Radiometric quality restoration can be defined as the detection and removal of radiation anomalies recorded by a sensor, with a view to maintain a range of brightness suitable for the imaged land features. Several studies have been and are still underway to devise radiometric approaches for enhancing the quality of UAV images [
10,
13,
14,
15].
Although radiometric errors in aerial imagery are a function of the local image contrast, tonal array, random noise, and radiometric resolution [
16], the presence of shadows on croplands tends to play a role in the alteration of radiometric properties of crops. This subsequently limits the retrieval of crop-related information. This is true, despite the substantial amount of beneficial landscape feature information that a shadow provides, such as shape, relative position, height [
17,
18], and illumination direction [
19]. A shadow makes image pixels darker, and this may lead to noticeable alteration in spectral patterns of crops. Subsequently, a partial or a complete loss of spectral information regarding crop types and condition may occur [
20,
21]. The occluded crops may subsequently be subjected to spectral confusion and misclassification, and this may have serious implications on agricultural-monitoring programs and the attainment of global food security and hunger alleviation goals. At the crop level, occlusion may occur as a result of the presence of cast- and self-shadows. Whereas a cast shadow occurs due to the blocking of light by another object at the scene, a self-shadow occurs due to occlusion by the crop itself, i.e., the side of the crop itself that is not exposed to the illumination source [
22]. This may also occur when the top-most leaves of a crop occlude the leaves situated below them. Therefore, occluded crops must be radiometrically compensated for their accurate and precise characterization. A reliable radiometric compensation approach must be able to remove cast shadows while preserving self-shadows as part of the crop. Occluded crops can be radiometrically compensated via detecting shadow and removing the distortions from imagery, each of which can be investigated independently of the other [
23]. In fact, the techniques used to radiometrically compensate occluded pixels from remote-sensing images can be achieved through two main steps: detection of shadows and performing a de-shadowing process [
24].
Radiometric compensation of crops under occlusion can be accomplished using a thresholding algorithm, modeling, or object-oriented techniques. The thresholding process involves determining the optimal threshold value of a digital number by analyzing histograms to differentiate shadow information from other types of information [
25]. Modeling approaches are commonly used since they are less complex than thresholding, which requires prior knowledge of shadow and mathematical modeling. On the other hand, an object-oriented technique can also effectively detect shadows but does not operate directly on individual pixels, as it operates on image fragments [
26]. However, modeling and object-oriented approaches are complicated, and they involve a long sequence of equations [
26]. Although thresholding has demonstrated the capability to radiometrically compensate occluded regions, this approach is only suitable for compensating regions under partial occlusion [
27,
28]. Moreover, studies on radiometric restoration have extensively focused on built-up environments, neglecting other environmental disciplines [
19,
28,
29,
30,
31]. We envisage that, when used in conjunction with a brightness-tuning approach, thresholding can aid in the restoration of radiometric properties of crops under total occlusion. Owing to the limited ability to effectively handle total occlusion scenarios and the lack of simplicity and reliability of these radiometric compensation techniques, this study proposes a simple and reliable approach for radiometrically compensating crops under total occlusion by integrating brightness-based compensation and thresholding techniques. This approach relies on tuning brightness properties of the imagery while observing the restoration patterns of spectral information in the occluded region.
2. Material and Methods
2.1. Experimental Site Characterization
The experimental site was situated within the Mutale River catchment in the Limpopo province of South Africa, which is well known for its agricultural practices. Small-scale crop farming for supporting local livelihoods and the rural economy is dominant in this area. The small-scale farms of the study area were found at 22°47′37.22″ S, 30°29′08.41″ E absolute location of the Earth. A subtropical climate, with a mean annual rainfall ranging between 300 mm and 1000 mm, characterized the experimental site [
32], with a large amount of rainfall received during the summer season. The experimental site was a part of many small land plots that cultivated a variety of crops, such as maize, cabbage, sweet potatoes, sugar beans, peas, green beans, and
Solanum retroflexum, across all the seasons due to an available furrow irrigation system to support crop growth during rain-scarce seasons. The variety of crops cultivated in this area underscores the need to quantify these crops for yield estimation purposes.
Figure 1 shows the location of the experimental site with respect to South Africa and the Mutale River catchment. The selection of the experimental site was prompted by the presence of a shadow, which appears as large black spot in the image, created by an adjacent tree.
2.2. Methods
The following sequence of methods and techniques was carried out to achieve the purpose of this study (
Figure 2).
UAV Imagery Acquisition
Remotely sensed imagery used in the current study was acquired from a UAV platform. Prior to the acquisition of the UAV imagery, the weather conditions were assessed to ensure that the images were acquired during midday and in cloudless conditions to eliminate the influences of haze, smoke, and clouds on the quality of the imagery. The UAV imagery was acquired on 30 June 2021. Two drone flight campaigns were surveyed to capture images during midday (i.e., between 12:00 and 14:00). A DJI Matrice 600 UAV with a Mica Red-Edge Multispectral Sensor captured land surface images using an 8.5 cm spatial resolution. This UAV multispectral sensor captured images in five spectral channels of the electromagnetic spectrum, i.e., red, green, blue, near-infrared, and red-edge. The flight-imaging process was carried out at a speed of 10 m per second. The flight plan was designed such that the imaging process created a minimum lateral and frontal overlap of 75% for feature matching and mosaicking in postprocessing. The flight altitude for the UAV was set at 120 m above ground level, capturing images at a spatial resolution of 8.0 cm.
Table 1 provides details of the UAV multispectral sensor employed in the study area.
2.3. UAV Camera Calibration
Calibration of each camera of the UAV multispectral sensor was performed to remove the lens distortion and calculate the focal length and principal point of each camera. Each camera designed and captured a white chess-board target with different roll and pitch angles to produce convergent images. The multispectral camera was configured to ensure 75% overlap between consecutive images. This was carried out to ensure that all the portions of the study area were covered during the imaging process and to facilitate an accurate orthomosaic process. An image of a reflectance calibration panel was captured before and after each flight to remove the effects of sunlight variation and reflectance characteristics.
2.4. UAV Image Processing
The raw remotely sensed data collected by unmanned aerial vehicles represent the Earth’s irregular surface; therefore, georeferencing was utilized to assign map geographic coordinates to image data. In this study, geometric correction was used to ensure that pixels or features in an image were in their proper and exact position on the Earth’s surface and to minimize or reduce geometric distortions between sets of data points. This was achieved by employing the nearest neighbor resampling techniques in the TerrSet 18.31 geospatial-monitoring software package. Georeferencing is frequently used in the correction process because shifting pixels to remove distortion and assigning coordinates to those pixels can both be performed at the same time.
2.5. Image Stretching
An image enhancement technique in the form of linear stretching was applied to rescale the pixel values of each original image band to new values that ranged from 0 to 255. The linear stretching was performed on each UAV band using the Stretch module embedded in the TerrSet platform based on Equation (1):
IV is the value of a pixel in the input map,
ILL is the lower value of the “stretch from” range,
IUL is the upper value of the “stretch from” range,
OLL is the lower value of the “stretch to” range,
OUL is the upper value of the “stretch to” range.
The input values were determined by the ‘stretch from’ values, and the lower and upper ‘stretch from’ boundary values were involved in the stretching process. The output values were then determined using the output domain, the value range, and precision of this domain.
2.6. Conversion of Digital Number (DN) Values to Radiance
Multispectral images require conversion from digital number (DN) to reflectance data before they can be interpreted or used as input for image analysis [
33]. When reflectance maps are the intended end products, results can be improved by performing absolute radiometric sensor correction. This involves converting unitless DN values into at-sensor radiance
L using the following Equation (2):
such that
where
L(
x,
y) for each
x,
y pixel is determined in terms of
V(
x,
y), which is the vignetting correction of the normalized raw DN and normalized black-level DN [
34].
a1, a2, and a3 denote radiometric correction factors. The sensor-specific constants a1−3a may be obtained from the metadata file of the UAV imagery-scanning specification results.
p(x, y) and pBL(x, y) denote pixel radiance at location (x, y),
g denotes the sensor gain,
te denotes the exposure time.
The DN values were converted to calibrated radiance values based on the user-defined values for Lmin/Lmax for UAV sensor systems. This process was achieved using the image calculator embedded in the TerrSet 18.31 software package.
2.7. Image Filtering
Shadow has demonstrated the ability to produce sharp discontinuities in an image. These discontinuities are abrupt changes in pixel intensity, which characterize boundaries of objects in a scene [
35]. In this study, edge pixels were detected via applying a filtering technique in the form of Gaussian-based Sobel edge detection. This technique uses a pair of 3 × 3 convolution kernels or masks,
Gx and
Gy, as shown in Equation (4) adopted from Yin et al. [
36]:
where
G denotes the Gaussian filter,
x is the resulting image after applying the kernel Kx to the input image,
y is the resulting image after applying the kernel Ky to the input image.
Kernels
Kx and
Ky were computed using the matrices in Equation (5) and Equation (6), respectively:
These convolution kernels are typically fused together to determine the absolute magnitude and the
x- and
y-orientations (horizontal and vertical directions) of the gradient. The selection of the Sobel operator was based on its insensitivity to noise and relatively small mask in image detection [
37].
2.8. Demarcation of Shadow
Demarcation of shadow at the experimental site was achieved using the brightness statistics of each band of RGB. A total of 100 points were randomly digitized on the occluded region and superimposed on each spectral band of the UAV imagery. The pixel values on which the points were overlain were then extracted to compute the brightness statistics of the occluded region under each spectral band. Upon the successful computation of the descriptive statistics, the mean radiance value of each spectral band was used as a threshold for demarcating shadow, such that
where
L is the radiance of
shadow,
i is the pixel at the
i-th location, and
j is the pixel at the
j-th location.
This process was carried out in an ArcMap GIS environment.
2.9. Derivation of Color Composite Image
A 24-bit RGB composite image was initially produced to facilitate the identification of shadow and various crops. This composite image showed features using the proportions of radiance reflected in three channels of the UAV sensor. A 24-bit false color composite (FCC) image was also generated based on the NIR, red, and red-edge spectral bands. The motive behind the FCC image generation was to ensure that the NIR and red-edge spectral channels to which crops are sensitive also underwent the radiometric restoration process.
2.10. Generation of Spectral Vegetation Indices for Occluded Crop Characterization
Several vegetation indices were generated for the purpose of evaluating their efficacy in characterizing occluded crops.
Table 2 provides a list of the spectral vegetation indices generated for the purpose of this study. The selection of these spectral vegetation indices was informed by the spectral resolution of the UAV sensor employed in this study.
Table 2.
List of spectral vegetation indices generated in this study.
Table 2.
List of spectral vegetation indices generated in this study.
Index | Equation | Author(s) |
NDVI | | Filgueiras et al. [38] |
GNDVI | | Mangewa et al. [39] |
SAVI | | Wang et al. [40] |
OSAVI | | Bastiaanssen et al. [41] |
GOSAVI | | Ji et al. [42] |
NDRE | | Crema et al. [43] |
2.11. Classification of UAV Imagery
Land cover types were classified using a supervised image classification technique. Initially, the training site was created for four (4) cover types, namely cabbage, maize, soil, and shadow. The inclusion of shadow as a land cover class was triggered by the inability of the composite images and spectral vegetation indices to identify and recognize crops and soil located in the shadowed region. Four supervised image classifiers, namely k-nearest neighbor, maximum likelihood, multilayer perceptron neural network, and object-oriented, were applied to categorize these land cover types based on the spectral bands prior radiometric compensation. This process was repeated for only three (3) land cover types, viz., cabbage, maize, and soil, based on a radiometrically compensated RGB image.
2.12. Radiometric Compensation
The initial step of radiometric restoration was to compensate the occluded region by adding the radiation of the occluded region (
Ldir(
P)) and the diffused radiance from the region adjacent the occluded region. This was achieved by modifying Equation (14) adopted from Li et al. [
44]:
where
Ldir(
P) denotes radiation directed to the sensor from the occluded region,
Ldiff denotes the diffused radiance from the adjacent region,
Latm denotes the radiance reflected by the atmosphere without reaching the ground.
The radiometric compensation Equation (14) adopted from Li et al. [
44] is relevant for compensating radiometric properties of occluded regions as imaged with satellite sensor. As such, the Equation also accounts for the radiation loss (
Latm) as a result of interaction with the atmosphere. In this study, the radiance reflected by the atmosphere (
Latm) was deliberately omitted because the employed UAV system captured the image while situated below the atmosphere to record the reflected radiation that did not interact with the atmosphere, such that
Subsequently, an iterative thresholding method was applied to further radiometrically compensate the occluded region of the study area by tuning brightness using both TCC and FCC images as inputs to the equation. The brightness tuning was achieved via obtaining the minimum and maximum radiance values of each spectral band fused in both the TCC and FCC images. The brightness of each image was set at the initial threshold (
T0) to divide each image into foreground and background using Equation (16):
where
T0 denotes the initial threshold,
radmax denotes the maximum radiance value of the image,
radmin denotes the minimum radiance value of the image.
The subsequent brightness values of the image were obtained via pushing the threshold to the next level, such that
where
k denotes the spectral channel to be radiometrically compensated, i.e., R, G, and B or NIR, R, and RE channels.
+1 denotes radiance values at the next threshold level.
This process was repeated until the brightness threshold reached its saturation, such that
where
Tk denotes the
k-th threshold.
The brightness threshold saturation was reached when Tk+1 = Tk, denoting the possible maximum threshold level of the image. However, four threshold levels were applied in this study. Upon the successful radiometric compensation process, the supervised image classification process was repeated with the exclusion of shadow as a land cover to determine the deviation in the area of the cover types under a radiometric compensation situation for those generated under uncompensated situation.
2.13. Spectral Radiance Evaluation of Compensated Land Features
The compensated radiometric properties of the occluded land features were evaluated to determine the extent to which they deviated from the same land feature types situated in the sunlit area of the experimental site. For this purpose, the relative error to the mean technique was applied to evaluate the radiance of both maize and soil using Equations (19) and (20) proposed by Thai et al. [
45]:
This method facilitated the measurement of the amount of radiance error in the restored radiance properties of the occluded land feature types relative to the radiance amounts of the sunlit land features.
4. Discussion
The importance of analyzing the spectral properties of land features has been well received across several disciplines, such as crop type and condition assessment [
46]. Shadow has the ability to cause a loss in radiometric information, leading to pixel misclassification and image misinterpretation [
47]. Shadows are an unavoidable component of high-resolution remotely sensed imagery, and the impact of shadows increases as the spatial resolution of imagery increases [
48]. Tomas et al. [
49] also noted that accurate extraction of pure crop and bare soil pixels is always challenging as a result of the influence of shaded pixels. This study was aimed at proposing a simple approach for radiometrically compensating crops under occlusion for their improved quantification. The UAV system facilitated the successful acquisition of high-spatial-resolution imagery for characterizing crops at the experimental site. Milas et al. [
50] noted UAV systems are capable of providing detailed information about land features, even at a resolution of several centimeters, and that removing shadow from data acquired using these systems is not easy. This is even more difficult if the imagery is to be subjected to image classification process, due to a challenge pertaining to the description of the distinct properties of various land feature classes using single-level features [
44]. Movia et al. [
47] also noted that, although high-spatial-resolution UAV images facilitate the retrieval of many land feature information types, they are subjected to classification problems as a result of shadow. In this study, dead pixels were noted when characterizing occluded land feature types with spectral vegetation indices. In order to achieve accurate and automatic crop detection, along with correct segmentation parameters, it is necessary to find an automatic and efficient method to look for the fourth-threshold value that sets the breakpoint between vegetation and bare soil. These pixels appear with no spectral radiance value, especially when characterizing occluded land feature types with the NDVI, SAVI, NDRE, and GNDVI spectral vegetation indices.
Although Milas et al. [
50] also noted a higher sensitivity of several classification algorithms to shadow at different spatial resolutions, their study attributed this sensitivity to variations in the texture of land features. Texture would not be properly identified and, subsequently, the shadow could neither be identified nor eliminated properly. In this study, the KNN classifier classified some soil as maize, whereas some maize cultivars were recognized as cabbage with the maximum likelihood classifier. This could be attributed to radiometric restoration in the imagery. In their study to remove shadow through separated illumination correction for urban aerial remote-sensing images, Luo et al. [
51] also noted that some road and vegetation fragments in the occluded regions were categorized as buildings. Moreover, several existing shadow removal methods evaluate their results quantitatively, as no shadow free-ground truth is available [
51]. No technique can deal with a shadow projected on a complex texture [
50]. The results from visual and statistical assessments indicated a significant difference between soil/vegetation indices in sunlit and shaded pixels [
52]. The areas of discontinuity between illuminated and occluded land features were not consistent across the experimental site due to variations in the illumination condition, as also noted in the study of Pons and Padró [
53]. The area coverage of maize was noted to be exaggerated as a result of the spectral similarity between maize and weeds located among cabbage. Moreover, the REM results of radiometrically compensated soil revealed the limitation of the proposed approach in completely restoring occluded soil. It is important to note that, during the radiometric compensation process, the land feature types in the sunlit area were also subjected to the process. This might have influenced the accuracy of the classification of land feature types after the radiometric compensation process. Moreover, this study did not employ field-based measurements to verify the precision of the estimated areal coverage of occluded crops. As such, the reliability of the classifiers was evaluated via visual comparison of the classified land cover types with those shown in the TCC image. However, only ground-based knowledge was employed in the recognition of land feature types without measurements.