Next Article in Journal
A Comparative Analysis of RNAi Trigger Uptake and Distribution in Mosquito Vectors of Disease
Previous Article in Journal
Survey Results of Honey Bee Colony Losses in Winter in China (2009–2021)
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Rapid Assessment of Insect Pest Outbreak Using Drones: A Case Study with Spodoptera exigua (Hübner) (Lepidoptera: Noctuidae) in Soybean Fields

1
Entomology Program, Division of Plant and Soil Sciences, West Virginia University, Morgantown, WV 26506, USA
2
Department of Plant Protection, Faculty of Agriculture, University of Tabriz, Tabriz 5166614888, Iran
3
Crop Foundation Division, National Institute of Crop Science, Rural Development Administration, Wanju 55300, Republic of Korea
4
Division of Crop Protection, National Institute of Agricultural Sciences, Rural Development Administration, Wanju 55300, Republic of Korea
*
Author to whom correspondence should be addressed.
Insects 2023, 14(6), 555; https://doi.org/10.3390/insects14060555
Submission received: 1 May 2023 / Revised: 28 May 2023 / Accepted: 12 June 2023 / Published: 15 June 2023
(This article belongs to the Topic Integrated Pest Management of Crops)

Abstract

:

Simple Summary

When an insect pest outbreak occurs, it is crucial to quickly assess the damage to manage the outbreak effectively. This study investigated a serious outbreak of the beet armyworm, Spodoptera exigua (Hübner) (Lepidoptera: Noctuidae), that occurred in soybean fields in South Korea. We conducted an aerial survey of 31 soybean blocks within the outbreak region using drones. The aerial images were analyzed to quantify soybean defoliation and to investigate the spatial patterns of the soybean damage by S. exigua. The results of this study showed that the aerial survey was an effective and rapid method for estimating the defoliation of soybeans caused by S. exigua. Moreover, it was found that the aerial survey followed by image analysis was more economical and required less time than a conventional ground survey, especially when the number of target soybean blocks subject to the survey was more than 15 blocks. Overall, the study demonstrated the effectiveness of using an autonomous drone and image analysis to conduct a low-cost aerial survey of soybean damage caused by S. exigua during its outbreak.

Abstract

Rapid assessment of crop damage is essential for successful management of insect pest outbreaks. In this study, we investigated the use of an unmanned aircraft system (UAS) and image analyses to assess an outbreak of the beet armyworm, Spodoptera exigua (Hübner) (Lepidoptera: Noctuidae), that occurred in soybean fields in South Korea. A rotary-wing UAS was deployed to obtain a series of aerial images over 31 soybean blocks. The images were stitched together to generate composite imagery, followed by image analyses to quantify soybean defoliation. An economic analysis was conducted to compare the cost of the aerial survey with that of a conventional ground survey. The results showed that the aerial survey precisely estimated the defoliation compared to the ground survey, with an estimated defoliation of 78.3% and a range of 22.4–99.8% in the 31 blocks. Moreover, the aerial survey followed by image analyses was found to be more economical than the conventional ground survey when the number of target soybean blocks subject to the survey was more than 15 blocks. Our study clearly demonstrated the effectiveness of using an autonomous UAS and image analysis to conduct a low-cost aerial survey of soybean damage caused by S. exigua outbreaks, which can inform decision-making for S. exigua management.

1. Introduction

Rapid surveys and assessments of insect pest outbreaks are critical for timely management of decision-making and response to the outbreaks. Ground surveys and aerial sketch mapping by experts on small airplanes have traditionally been used for large-scale surveys of pest outbreaks in field crops and forests, respectively. When the damage is extensive, satellite images may be used to assess the outbreak. However, aerial sketch mapping can be prone to subjectivity error, and remote sensing using satellites has low spatial resolution, making it difficult to directly detect and assess pest outbreaks when damage is scattered across a large geographic area. For example, the United States Department of Agriculture (USDA) Forest Service uses a combination of field surveys and digital aerial sketch mapping, but this method can be influenced by factors such as the experience of the sketch mapper, types of signs and symptoms, aircraft type, and flight speed [1]. Remote sensing using satellites has been used for surveying insect pest outbreaks but has limited applicability in operational pest-management programs [2] due to low image resolution. As a result, there is a need for alternative methods such as unmanned aerial systems and image analysis to conduct low-cost and rapid surveys of pest outbreaks, which can inform timely and effective management decisions.
Unmanned aircraft systems (UAS), commonly known as drones, have become an attractive tool for aerial surveys of crop stress and damage caused by pests [3,4]. UAS flights over agricultural lands or forests require fewer logistics and field crew members compared to manned airplanes [5]. Recent advances in image processing and analysis tools, coupled with high-resolution imagery obtained by UAS, provide a unique opportunity for field image processing capability [6]. UAS have several advantages compared to manned airplanes and satellite imagery for monitoring crop stress and damage, including safety, cost, flexibility, and modularity. Unlike satellite imagery, UAS imagery can be acquired in real-time and tailored to meet site-specific survey and management strategies by locating specific areas that require additional monitoring [6]. The modularity of onboard payloads, combined with an onboard control system, allows for easy retrofitting with a wide range of high-resolution imaging payloads. Recent significant advances in miniaturized flight control technology and expanded capabilities of UAS for autonomous flight offer the possibility of conducting aerial surveys repeatedly and even without human intervention [7].
The beet armyworm, Spodoptera exigua (Hübner) (Lepidoptera: Noctuidae), is a key pest of vegetables, field crops, and floricultural crops, with over 90 species of plants in 18 families being reported as host plants globally [8,9,10,11]. The damage by S. exigua is predominantly caused by larvae that feed on leaves resulting in feeding signs of defoliation. Outbreaks of S. exigua could be associated with weather patterns [9,12], its short life cycle (16–42 days from egg to adult), high fecundity (700–1300 eggs per female), number of generations per year [13,14,15], its spatial distribution pattern [16], and large-scale migration of adults [12,17]. In soybean fields, we observed heavy damage by S. exigua causing nearly 100% defoliation.
Current recommendations for S. exigua control include counting the number of larvae and applying insecticides as needed; the economic threshold for chemical control is 6.3 larvae/plant for cabbage [18]. Chemical control is the most common practice to control S. exigua in South Korea, but the strong insecticide resistance of S. exigua has made the control of the moth extremely difficult [19,20]. Specifically, while first and second instars are relatively susceptible to insecticides, third to fifth instars are tolerant to the insecticides, and they generally hide within plants and thus lower the chance of exposure to foliar-applied insecticides. Biological control agents including the entomopathogenic fungus Nomuraea rileyi [21] and a nuclear polyhedrosis virus (NPV) have been identified [22] but are currently not commercially available. Therefore, early detection of damage by S. exigua is considered an ideal preventative control measure. Although early detection of S. exigua might be possible by using doppler weather radar when the swarm of S. exigua adults migrates during a major outbreak [23,24], the technology is not readily available to growers and pest control service providers.
In July and August of 2018, an outbreak of S. exigua occurred in soybean fields (Glycine max [L.]) in Jeollabuk-do, the western region of South Korea, covering an area of 1701 ha near the cities of Kimje, Gunsan, and Iksan. An emergency ground survey was conducted by the government agencies of South Korea to estimate the soybean damage caused by S. exigua and to make management decisions such as applying pesticides or reseeding soybean or alternative crops. Taking advantage of the outbreak, we conducted an aerial survey with UAS simultaneously with the ground survey. Our hypothesis was that the aerial survey of S. exigua damage using UAS and image analysis could be more efficient than typical ground-based visual surveys. The objectives of this study were (1) to develop a protocol for a rapid assessment of S. exigua outbreaks in soybean fields using rotary-wing UAS and image analysis, (2) to compare the efficiency of aerial surveys with conventional ground surveys, (3) to determine the spatial pattern of S. exigua damage using spatial statistics, and (4) to check the applicability of satellite imagery for detecting soybean damage caused by S. exigua at a regional scale.

2. Materials and Methods

2.1. S. exigua Outbreak and Survey Site

This study was conducted in Juksan Township (N 35.759012, W 126.789840) close to the city of Kimje where the most serious outbreak of S. exigua was reported during the 2018 outbreak in South Korea. An initial field visit by the entomologists at the National Institute of Agricultural Sciences of South Korea confirmed that the larvae feeding on soybean foliage were exclusively S. exigua (Figure 1).

2.2. Ground Survey and Damage Assessment

A team of 19 field-survey experts from interagency organizations, including the National Institute of Agricultural Sciences, Province Agricultural Research and Extension Station, and Agricultural Extension and Technology Center, conducted a conventional ground survey to estimate the damage caused by S. exigua in the study site for a rapid response to the outbreak. The survey involved selecting 30–50 soybean plants in each soybean block and rating the damage using five categories: severe (75–100% of soybean damaged), high (50–75%), medium (25–50%), low damage (1–25%), and no damage (0%). The location of soybean blocks surveyed, time taken by 19 surveyors, and damage estimation were recorded.

2.3. Aerial Survey and Damage Assessment

A three-step protocol for assessing damage in soybean fields caused by S. exigua was developed with the goal of efficiently completing an aerial survey (Figure 2). The first step involved deploying a rotary-wing UAS (DJI Phantom 3 Advanced; SZ DJI Technology Co., Ltd., Shenzhen, China) to capture aerial imagery. The second step involved downloading the aerial images and stitching them together to generate a geo-referenced composite image of the study site. In the third step, image analyses were conducted to estimate the amount of damage caused by S. exigua.
The UAS was equipped with an RGB camera capable of both photography and videography at a resolution of 1800 p. During the UAS flight, a series of aerial photographs were taken with 80% image overlap between two consecutive images taken. Two different aerial survey methods were employed: (1) a high-altitude survey in autonomous flight mode conducted at 50 m above the ground, and (2) a low-altitude spatially targeted survey conducted at 5–10 m above the ground to confirm S. exigua damage on individual soybean plants in each block (Figure 3).

2.4. Image Processing and Analysis

Following the completion of the flight missions, the aerial images were downloaded from the UAS. For the high-altitude survey, the images were aligned according to the pre-planned flight path (Figure 3A) and processed using Pix4DMapper software (Pix4D, Prilly, Switzerland) to generate a geo-referenced composite image. The composite image was then ortho-mosaicked with ArcInfo® 10 software (ESRI, Redland, CA, USA) to confer spatial attributes to the map coordinate system. Before conducting the image analysis to estimate the damage caused by S. exigua, a validation survey using a low-altitude flight was performed to confirm that the damage was indeed caused by defoliation.
To assess the extent of defoliation caused by S. exigua in each soybean block, ArcInfo and Photoshop CS4 (Adobe Inc., San Diego, CA, USA) were used to analyze the composite image. The green pixels in the image were counted to estimate the amount of soybean foliage that remained after S. exigua feeding. This method was calibrated by comparing the number of pixels counted in an undamaged area and by calculating the percent reduction due to defoliation based on the number of green pixels present in the damaged area.

2.5. Determining the Efficiency of the Aerial Survey with UAS

To determine the efficiency of the aerial survey with UAS, a cost analysis was conducted. Based on flight information recorded in the UAS, we extracted the time of UAS operation for both high- and low-altitude aerial surveys (TUAS). Additionally, time for image processing and analysis (Tanalysis), the number of soybean blocks surveyed by UAS flights (Nblock), the expert labor fee per hour for the UAS pilot and image analyst (Fexpert), and the technology fee per hour (Ftechnology) were recorded. We set USD 50 per hour for an expert fee and USD 100 for Ftechnology based on average commercial rental rates (i.e., USD 50 for UAS rental and USD 50 for software rental). The total cost for an aerial survey with UAS (Caerial) per soybean block was calculated as:
Caerial = [(TUAS + Tanalysis) × (Fexpert + Ftechnology)]/Nblock
To compare the aerial survey with the conventional ground survey, we calculated three variables associated with the ground survey: the number of soybean blocks to survey (Nblock), the time required for the ground survey (Tground), and the hourly labor fee (Flabor) of USD 10 per field surveyor. The total cost per block by a conventional ground survey (Conventional) per block was calculated as:
Cconventional = (Flabor × Tground)/Nblock
Regression analyses were conducted to determine the relationships between the cost and time for different numbers of soybean blocks to survey and those for each of the aerial and ground surveys. All the statistical analyses were conducted with SAS [25] at an error rate of 0.05.

2.6. Spatial Patterns of Soybean Damage by S. exigua

We conducted spatial analyses with geostatistics and spatial analysis by distance indices (SADIE) to characterize the spatial pattern and to test the significance of spatial aggregation of soybean damage. UAS images taken at 50 m above the ground were used for the analyses. Soybean blocks with >95% defoliation were excluded from the analyses, as they showed near complete defoliation. The images of soybean defoliation were reclassified into two classes (defoliated or healthy pixels) using ArcGIS Pro 10. Each soybean block was divided into 54-m-by-54-m grids, and the number of pixels representing soybean defoliation in each grid was counted to measure the amount of damage in the grid. The data were analyzed using geostatistics (i.e., semivariogram modeling) to quantify the degree of the spatial dependence and to characterize the spatial structure [26] of soybean damage. Geostatistical analyses were conducted using GS+ 10 (Gamma Design Software, Plainwell, MI, USA). The best theoretical semivariogram models were selected based on minimum residual sums of squares (RSS) and maximum r2 values. Three semivariogram parameters were used to describe the spatial structure of soybean damage by S. exigua: range, sill, and nugget [27]. Range is the distance at which the semivariance reaches a maximum and represents the lag distance beyond which samples are spatially independent. The sill is the value of the semivariance at any distance greater than or equal to the range. The nugget is the value of the semivariance when lag distance equals zero. By using the these parameters, the degree of spatial dependence (DD) was calculated with the following formula:
D D = C C 0 + C × 100
where C is sill and C0 is nugget. Spatial dependency is considered weak, moderate, and strong when DD is ≤25%, 26–75%, and ≥76%, respectively [28].
Although semivariograms can quantify spatial dependency, they do not determine spatial clustering or aggregation. Therefore, SADIE was used to test the statistical significance of spatial aggregation or clustering of soybean damage by S. exigua under the null hypothesis of complete randomness [29]. The spatial pattern was determined by calculating the index of aggregation (Ia), and the statistical significance of aggregation was determined with associated probability (Pa). A value of Ia = 1 indicates a random spatial distribution, Ia > 1 reveals an aggregated pattern, and Ia < 1 is indicative of a regular or uniform spatial distribution. The aggregation was considered significant when Pa < 0.05. Overall spatial clustering was tested by calculating mean clustering indices ( v i and v j ) and their associated probabilities, P v i and P v j . A value of P v i < 0.05 confirmed the presence of significant spatial clustering into patches and P v j < 0.05 indicated a high level of spatial clustering into gaps [30]. These analyses were conducted by using SADIEShell version 2.0 (Rothamsted Experimental Station, Harpenden Herts, United Kingdom).

2.7. Regional-Scale Soybean Damage by S. exigua

To identify soybean fields damaged by S. exigua in the outbreak region of South Korea, we analyzed time series satellite data for change detection. We utilized the normalized difference vegetation index (NDVI) before, during, and after the S. exigua outbreak. The NDVI is a measure that detects and quantifies the presence of live green vegetation by analyzing reflected light in the visible and near-infrared (NIR) bands. Consequently, NDVI values can serve as indicators of the health or condition of vegetation within each pixel of a satellite image.
We used Landsat 8 Operational Land Imager/Thermal Infrared Sensor (OLI/TIRS) Level 2 imagery with a spatial resolution of 30 m to calculate NDVI values before, during, and after the S. exigua outbreak within the study site region. Specifically, we downloaded Landsat 8 OLI/TIRS Level 2 imagery with 11 spectral bands from the U.S. Geological Survey Earth Explorer for Path 115, Row 035. The NDVI values were calculated using the equation: NDVI = (NIR − Red)/(NIR + Red), where Red represents band 4 and NIR represents band 5.
To identify regions affected by S. exigua damage, we subtracted the NDVI values during the damage from those before the damage using a raster calculator. Similarly, we subtracted the NDVI values during the damage from those after the damage to locate regions where regrowth or replanted crops occurred within the damaged area.

3. Results

3.1. Ground Survey and Damage Assessment

A ground survey by experts within the outbreak area showed 53.3%, 31.1%, 13.3%, and 2.2% severe, high, medium, and low damage to soybean plants, respectively. The ground survey also found that S. exigua larvae feeding on the soybean plants were mostly second and third instars with fewer fourth instars, indicating the damage was in progress, and thus urgent control of S. exigua larvae was needed. The surveyors also observed and reported that soybean blocks planted late experienced higher damage, and S. exigua larvae had completely defoliated soybean plants, including stems, in some soybean blocks, causing nearly 100% crop loss (Figure 4).

3.2. Aerial Survey and Image Analysis

A total of 150 aerial images were taken during the high-altitude UAS flight (Figure 3A). A low-altitude flight confirmed soybean defoliations in all 31 soybean blocks (Figure 4A). The composite image (Figure 5) showed various levels of soybean damage by S. exigua. Specifically, soybean blocks 10, 11, and 12 were completely defoliated by S. exigua (Figure 5 and Figure 6, Table S1).
The analysis of the aerial images showed that each pixel in the composite image corresponded to an area of 0.02 m2, roughly equivalent to one soybean plant. This allowed for estimation of plant-level defoliation. The results indicated that S. exigua feeding had removed 78.3% of soybean foliage in the study site; the total area of soybean fields planted was 10.11 ha, of which 7.92 ha were defoliated. The level of defoliation varied across soybean blocks, with the least and most severely affected blocks showing defoliation rates of 22.4% and 99.9%, respectively (Table S1 and Figure 7).

3.3. Economic Analysis to Determine the Efficiency of Aerial Survey

The ground survey of 45 soybean blocks was completed by 19 surveyors in 1.1 h per surveyor (i.e., Fexpert was set as 1.1). A pre-flight check, UAS inspection, and test flight were conducted before any UAS flight mission to ensure safe operation, taking 30 min. Both the high- and low-altitude aerial surveys took 57 min covering 20 ha. Therefore, the total time for UAS operation (TUAS) was 0.5 h for pre-flight check and 0.05 h per block. It took 2 h to finish image analysis for the 31 soybean blocks, and thus Tanalysis was set as 0.0645 h per block. Using these parameter values, the economic analysis revealed that the aerial survey with UAS was more time efficient than the ground survey when more than one soybean block was surveyed (Figure 8A). In addition, the cost for the survey was a more economical aerial survey with UAS than that by ground surveys when more than 15 soybean blocks are surveyed (Figure 8B).

3.4. Spatial Patterns of Soybean Damage by S. exigua

The geostatistical analyses revealed that non-linear models, such as exponential, spherical, and Gaussian models, provided the best fit for the data, indicating spatial correlation of soybean damage caused by S. exigua. The degree of spatial dependence (DD) was greater than 75% in 10 of the 13 soybean blocks and 50–75% in 3 of the blocks (Table 1), indicating an overall strong spatial dependence of S. exigua damage. The range values of the semivariograms, which represent the maximum distance at which values are spatially correlated, ranged from 89 m to 872 m for the soybean blocks we surveyed.
SADIE results revealed that soybean damage by S. exigua was spatially aggregated or clustered; Ia was > 1 in all 13 blocks (Table 2). The aggregated distribution of soybean damage was statistically significant in 12 blocks (Pa < 0.05). Strong clustering into gaps and patches ( v i > 1.5 and v j < −1.5) verified significant spatial aggregation of soybean damage in the 12 soybean blocks (P v i < 0.05 and P v j < 0.05).

3.5. Regional-Scale Soybean Damage by S. exigua

The NDVI values before (Figure 9A), during (Figure 9B), and after (Figure 9C) the S. exigua outbreak exhibited significant changes. The differences in NDVI values between before and during the S. exigua outbreak ranged from 0.058 to 0.683 (Figure 9D), indicating vegetation losses resulting from defoliation. Similarly, the NDVI values between during and after the S. exigua outbreak ranged from 0.066 to 0.582, corresponding to fields where soybeans had been replanted or had regrown following the S. exigua outbreak (Figure 9E,F). Within our study sites (outlined with a black polygon in Figure 9), 20 out of 30 soybean blocks were identified as damaged or exhibiting regrowth/replanting based on the NDVI values, while the defoliation in the other 10 blocks was not detected through the changes in NDVI values before, during, and after the S. exigua outbreak (Table S2).

4. Discussion and Conclusions

UAS have been increasingly utilized for various agricultural applications, such as crop and soil monitoring [31,32,33,34,35], pest detection [3,36], aerial releases of natural enemies [7,37], and pesticide application [38]. With the availability of small rotary-wing UAS equipped with high-resolution cameras, it is now possible to detect insects directly from aerial images [34]. Such high image resolution was also observed in our study (see Figure 1B for visible feeding signs on a soybean leaf). This level of resolution could be utilized to detect soybean damage caused by S. exigua at later crop stages, such as the flowering stage, even with lower levels of defoliation. However, remote sensing with UAS and precision agriculture are data-intensive procedures that often require skilled personnel [39]. In addition, automated detection of insect damage through image analysis is needed to make this technology more accessible to general agricultural practitioners. Our study demonstrated large-scale detection and rapid assessment of S. exigua damage using UAS.
Another technology that can be used for the detection of the S. exigua outbreak is satellites. Given the extensive damage to soybeans caused by the S. exigua outbreak, time series data containing spectral change information obtained from satellite imagery can be utilized to map insect damage at the regional level [40]. Our post-hoc analysis showed that the differences in NDVI values between before and during the S. exigua outbreak indicated vegetation losses due to defoliation. Similarly, the NDVI values between during and after the S. exigua outbreak coincided with fields where soybeans had been replanted or regrown after the S. exigua outbreak (Figure 9E). These results suggest that satellite data can be used to identify the hotspots of soybean defoliation in a large area with high levels of damage. However, it is still difficult to precisely locate individual soybean blocks with defoliation by S. exigua by using satellite and the changes in NDVI values before, during, and after the outbreak. In addition, the quality of Landsat 8 images is dependent on weather conditions, and the resolution is moderate when calculating NDVI using a moderate resolution imaging spectroradiometer (MODIS) sensor on satellites. As a result, obtaining suitable images for calculating NDVI values during a short pest outbreak period can be challenging. Our study is the first to detect the outbreak of S. exigua using NDVI, although a few previous studies have successfully demonstrated the potential of satellite-based NDVI in detecting outbreaks of the fall armyworm S. frugiperda in cornfields [40,41].
The results of this study indicate that aerial surveys using small rotary-wing UAS offer several advantages for detecting and managing insect outbreaks. One of the major advantages is that aerial surveys with UAS are often more cost-effective compared to conventional ground-based surveys. In the case of the S. exigua outbreak in South Korea, our study demonstrated that using UAS was more economical than conducting a ground survey when the number of target soybean blocks exceeded 15. This is because the UAS can cover a larger area in a shorter amount of time and with fewer personnel, thereby reducing survey costs. Another advantage of aerial surveys is that it can provide valuable information in the form of maps that show the distribution of insect damage. These maps can be used to guide where to conduct intensive surveys and management in the outbreak area, as well as to inventory the areas of damage. A digital inventory of insect outbreaks can be a useful resource for predicting and preventing future outbreaks, especially in the case of pests such as S. exigua, which have a history of occurring and damaging a wide variety of high-value crops periodically in South Korea since the first outbreak was reported in 1926 [42].
Site-specific management of S. exigua is another benefit of using UAS for outbreak surveys. By identifying the locations of damage, management measures can be applied only where needed [43], reducing pesticide inputs and increasing control efficiency [43,44]. Our study revealed that soybean damage caused by S. exigua exhibited spatial aggregation, and a previous study [16] also demonstrated the spatial aggregation of the larval stage of S. exigua. Therefore, UAS can collect detailed spatial data through aerial images, enabling the identification of hotspots and the spatial distribution of pest damage. This information can contribute to precise pest management practices and a reduction in sampling costs.
Additionally, our study found that the amount of defoliation by S. exigua varied considerably between adjacent soybean blocks (Figure 6C–E), suggesting that changing soybean variety or planting time could help avoid or reduce damage. This study showed that a major cost of using a small UAS for insect outbreak surveys was fees for the use of technology, including UAS and image analysis. As the price of such technology has decreased with technological advances [7] the use of UAS and image analysis could be cheaper in the future.
Finally, advances in technology have made UAS and image analysis more affordable, as well as the use of learning-based methods [45]. More recently developed convolutional neural networks [46,47] could further improve automated image processing and outbreak detection capabilities in the future. Artificial intelligence and machine learning would allow for more accurate and efficient monitoring and management of insect outbreaks. Specifically, deep learning in artificial intelligence (e.g., Mask2former model) is one of the potential tools for visual recognition of target pests [6], which could be used for early detection of insect pest outbreaks.
In conclusion, our study provided clear evidence of the effectiveness of utilizing an autonomous drone and image analysis for conducting a cost-effective aerial survey of soybean damage caused by S. exigua during an outbreak. The integration of UAS and image analysis proved to be an efficient and economical approach for assessing the extent of soybean damage caused by S. exigua. By swiftly assessing the magnitude of outbreaks and identifying areas requiring urgent management, rapid responses to further damage can be achieved.
Furthermore, our study involving UAS and satellite data demonstrated that precise assessments of pest outbreaks can facilitate the development of targeted pest management strategies, such as site-specific pest management [43]. The utilization of remote sensing through UAS and sensor technologies has become an essential method for site-specific pest management, as evidenced in our study. We discovered significant spatial aggregation of soybean damage caused by S. exigua and observed spatial autocorrelation across a wide area, highlighting the importance of employing this approach for effective pest management interventions.

Supplementary Materials

The following Supporting Information can be downloaded at: https://www.mdpi.com/article/10.3390/insects14060555/s1, Table S1: Damage of soybean by S. exigua in 31 blocks surveyed by UAS; Table S2: Comparison of change detection for NDVI values identified blocks and field damage.

Author Contributions

Conceptualization, B.Y.S., G.-S.L. and Y.-L.P.; methodology, B.Y.S. and Y.-L.P.; formal analysis, R.K., K.N. and Y.-L.P.; ground survey, B.Y.S., G.-S.L.; aerial survey, Y.-L.P.; investigation, K.N., G.L., R.K. and Y.-L.P.; resources, B.Y.S., G.-S.L. and Y.-L.P.; writing—original draft preparation, Y.-L.P.; writing—review and editing, all authors; visualization, K.N. and Y.-L.P.; supervision, B.Y.S.; project administration, B.Y.S., G.-S.L. and Y.-L.P.; funding acquisition, B.Y.S. and Y.-L.P. All authors have read and agreed to the published version of the manuscript.

Funding

The Brain Pool Program of the National Research Foundation of Korea (2018H1D3A2001526) and the Hatch Project (WVA00724) of West Virginia University Agriculture and Forestry Experiment Station supported this research.

Data Availability Statement

Data will be available upon request.

Acknowledgments

We thank K. J. Lee at the Rural Development Administration of South Korea, N. J. Choi at the National Institute of Crop Science of South Korea, and their team members for their help with the field survey.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. De Beurs, K.M.; Townsend, P.A. Estimating the effect of gypsy moth defoliation using MODIS. Remote Sens. Environ. 2008, 112, 3983–3990. [Google Scholar] [CrossRef]
  2. Jepsen, J.U.; Hagen, S.B.; Høgda, K.A.; Ims, R.A.; Karlsen, S.R.; Tømmervik, H.; Yoccoz, N.G. Monitoring the spatio-temporal dynamics of geometrid moth outbreaks in birch forest using MODIS-NDVI data. Remote Sens. Environ. 2009, 113, 1939. [Google Scholar] [CrossRef]
  3. Rasmussen, J.; Nielsen, J.; Garcia-Ruiz, F.; Christensen, S.; Streibig, J.C. Potential uses of small unmanned aircraft systems (UAS) in weed research. Weed Res. 2013, 53, 242–248. [Google Scholar] [CrossRef]
  4. Torres-Sánchez, J.; López-Granados, F.; De Castro, A.I.; Peña-Barragán, J.M. Configuration and specifications of an unmanned aerial vehicle (UAV) for early site specific weed management. PLoS ONE 2013, 8, e58210. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  5. Park, Y.-L.; Cho, J.R.; Seo, B.Y. Detection of Monema flavescens (Lepidoptera: Limacodidae) cocoons using small unmanned aerial system. J. Econ. Entomol. 2021, 114, 1925–1933. [Google Scholar] [CrossRef]
  6. Valicharla, S.K.; Li, X.; Greenleaf, J.; Turcotte, R.; Hayes, C.; Park, Y.-L. Precision detection and assessment of ash death and decline caused by the emerald ash borer using drones and deep learning. Plants 2023, 12, 798. [Google Scholar] [CrossRef] [PubMed]
  7. Park, Y.-L.; Gururajan, S.; Thistle, H.; Chandran, R.; Reardon, R. Aerial release of Rhinoncomimus latipes (Coleoptera: Curculionidae) to control Persicaria perfoliata (Polygonaceae) using an unmanned aerial system. Pest Manag. Sci. 2018, 74, 141–148. [Google Scholar] [CrossRef]
  8. Greenberg, S.M.; Sappington, T.W.; Legaspi, B.C.; Liu, T.-X.; Setamou, M. Feeding and life history of Spodoptera exigua (Lepidoptera: Noctuidae) on different host plants. Ann. Entomol. Soc. Am. 2021, 94, 566–575. [Google Scholar] [CrossRef]
  9. Azidah, A.A.; Sofian-Azirun, M. Life history of Spodoptera exigua (Lepidoptera: Noctuidae) on various host plants. Bull. Entomol. Res. 2006, 96, 613–618. [Google Scholar] [CrossRef]
  10. Pearson, A.C. Biology, Population Dynamics, and Pest Status of the Beet Armyworm (Spodoptera exigua) in the Imperial Valley of California. Ph.D. Thesis, University of California, Riverside, CA, USA, 1982. [Google Scholar]
  11. Idris, A.B.; Emelia, O. Development and feeding behaviour of Spodoptera exigua (Lepidoptera: Noctuidae) on different food plants. J. Biol. Sci. 2001, 1, 1161–1164. [Google Scholar] [CrossRef]
  12. Xia-Lin, Z.; Cong, X.P.; Wang, X.P.; Lei, C.L. A review of geographic distribution, overwintering and migration in Spodoptera exigua Hübner (Lepidoptera: Noctuidae). J. Entomol. Res. Soc. 2011, 13, 39–48. [Google Scholar]
  13. Zheng, X.L.; Wang, P.; Cheng, W.J.; Wang, X.P.; Lei, C.L.; Heckel, D. Projecting overwintering regions of the beet armyworm, Spodoptera exigua in China using the CLIMEX model. J. Insect Sci. 2012, 12, 13. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  14. Prasad, Y.G.; Gayathri, M.; Prabhakar, M.; Jeyakumar, P.; Vennila, S.; Subba Rao, A.V.M.; Bhaskara Rao, I.; Rao, K.V.; Ramachandra Rao, G.; Rao, V.U.M. Population dynamics of Spodoptera litura outbreak on soybean vis a vis rainfall events. J. Agric. Meteorol. 2013, 15, 37–40. [Google Scholar]
  15. Wen, L.; Zhang, Y. Modelling of the relationship between the frequency of large-scale outbreak of the beet army worm, Spodoptera exigua (Lepidoptera: Noctuidae) and the wide-area temperature and rainfall trends in China. Acta Entomol. Sin. 2010, 53, 1367–1381. [Google Scholar]
  16. Salmani-Moghanlou, J.; Valizadegan, O.; Naseri, B. Spatial distribution pattern of Spodoptera exigua (Lepidoptera: Noctuidae) on sugar beet and advantage of site-specific spraying in the pest management. J. Entomol. Soc. Iran 2021, 41, 135–149. [Google Scholar]
  17. Navasero, M.V.; Navasero, M.M.; Cayabyab, B.; Ebuenga, M.D.; Candano, R.N.; Burgonio, G.A.S.; Bautista, N.M.; Aquino, E.M., Jr.; Gaspar, G.G. Investigation on the 2016 outbreak of the onion armyworm, Spodoptera exigua (Hubner) (Lepidoptera: Noctuidae), in onion growing areas in Nueva Ecija. Philipp. Entomol. 2017, 31, 151–152. [Google Scholar]
  18. Sacheon Agricultural Development and Technology Center (SADTC). 2007. Available online: http://m.4000nongwon.net/board/list.php?id=de_note&no=227&page=6&category=&Search=&search_word=&mode=view# (accessed on 20 April 2023).
  19. Capinera, J.L. Beet Armyworm, Spodoptera exigua (Hübner) (Insecta: Lepidoptera: Noctuidae); University of Florida Cooperative Extension Service, Institute of Food and Agricultural Sciences, EDIS: Gainesville, FL, USA, 1999. [Google Scholar]
  20. Chau, L.M. Integrated pest management: A strategy to control resistance of Spodoptera exigua and Helicoverpa armigera caterpillars to insecticides on soybean in the Mekong Delta. Pestic. Sci. 1995, 43, 255–258. [Google Scholar] [CrossRef]
  21. Hwang, J.H.; Park, B.R.; Kim, G.S.; Lee, S.G.; Kim, Y.G. Identification of an entomopathogenic fungus, Nomuraea rileyi ANU101, infecting the beet armyworm, Spodoptera exigua, in Korea. Kor. J. Mycol. 1995, 37, 139–143. [Google Scholar] [CrossRef] [Green Version]
  22. Gelernter, W.D.; Federici, B.A. Isolation, identification, and determination of virulence of a nuclear polyhedrosis virus from the beet armyworm, Spodoptera exigua (Lepidoptera: Noctuidae). Environ. Entomol. 1986, 15, 240–245. [Google Scholar] [CrossRef] [Green Version]
  23. Feng, H.Q.; Wu, K.M.; Cheng, D.F.; Guo, Y.Y. Radar observations of the autumn migration of the beet armyworm Spodoptera exigua (Lepidoptera: Noctuidae) and other moths in northern China. Bull. Entomol. Res. 2003, 93, 115–124. [Google Scholar] [CrossRef]
  24. Westbrook, J.K.; Eyster, R.S. Doppler weather radar detects emigratory flights of noctuids during a major pest outbreak. Rem. Sens. Appl. Soc. Environ. 2003, 8, 64–70. [Google Scholar] [CrossRef]
  25. SAS Institute. Using JMP Student Edition for Windows and Macintosh: The User’s Guide to Statistics with JMP Student Edition; SAS Institute: Cary, NC, USA, 2009. [Google Scholar]
  26. Shayestehmehr, H.; Karimzadeh, R. Geostatistical analysis of spatial distribution of Therioaphis maculata (Hemiptera: Aphididae) and coccinellid lady beetles (Coleoptera: Coccinellidae). J. Crop Prot. 2019, 8, 103–115. [Google Scholar]
  27. Isaaks, E.H.; Srivastava, R.M. Applied Geostatistics. Oxford University Press: New York, NY, USA, 1989. [Google Scholar]
  28. Grego, C.R.; Vieira, S.R.; Lourenção, A.L. Spatial distribution of Pseudaletia sequax Franclemlont in triticale under no-till management. Sci. Agric. 2006, 63, 321–327. [Google Scholar] [CrossRef]
  29. Perry, J.N. Spatial analysis by distance indices. J. Anim. Ecol. 1995, 64, 303–314. [Google Scholar] [CrossRef]
  30. Perry, J.N.; Winder, L.; Holland, J.M.; Alston, R.D. Red–blue plots for detecting clusters in count data. Ecol. Lett. 1999, 2, 106–113. [Google Scholar] [CrossRef]
  31. Mulla, D.J. Twenty five years of remote sensing in precision agriculture: Key advances and remaining knowledge gaps. Biosyst. Eng. 2013, 114, 358–371. [Google Scholar] [CrossRef]
  32. Xiang, H.; Tian, L. Development of a low-cost agricultural remote sensing system based on an autonomous unmanned aerial vehicle (UAV). Biosyst. Eng. 2011, 108, 174–190. [Google Scholar] [CrossRef]
  33. Zhang, C.; Kovacs, J.M. The application of small unmanned aerial systems for precision agriculture: A review. Prec. Agric. 2012, 13, 693–712. [Google Scholar] [CrossRef]
  34. Daponte, P.; De Vito, L.; Glielmo, L.; Iannelli, L.; Liuzza, D.; Picariello, F.; Silano, G. A review on the use of drones for precision agriculture. In IOP Conference Series: Earth and Environmental Science; IOP Publishing: Bristol, UK, 2019; Volume 275, p. 012022. [Google Scholar]
  35. Mogili, U.R.; Deepak, B.B.V.L. Review on application of drone systems in precision agriculture. Proc. Comp. Sci. 2018, 133, 502–509. [Google Scholar] [CrossRef]
  36. Park, Y.-L.; Lee, D.-H.; Cho, J.R.; Seo, B.Y.; Cho, Y.-S. Advances, limitations, and future applications of aerospace technology for apple IPM. Kor. J. Appl. Entomol. 2021, 60, 135–143. [Google Scholar]
  37. Kim, J.; Huebner, C.; Reardon, R.; Park, Y.-L. Spatially-targeted biological control of mile-a-minute weed using Rhinoncomimus latipes (Coleoptera: Curculionidae) and an unmanned aerial system. J. Econ. Entomol. 2021, 114, 1889–1895. [Google Scholar] [CrossRef] [PubMed]
  38. Garre, P.; Harish, A. Autonomous agricultural pesticide spraying UAS. In IOP Conference Series: Materials Science and Engineering; IOP Publishing: Bristol, UK, 2018; Volume 455, p. 012030. [Google Scholar]
  39. Tsouros, D.C.; Bibi, S.; Sarigiannidis, P.G. A review on UAV-based applications for precision agriculture. Information 2019, 10, 349. [Google Scholar] [CrossRef] [Green Version]
  40. Zhang, J.; Huang, Y.; Yuan, L.; Yang, G.; Chen, L.; Zhao, C. Using satellite multispectral imagery for damage mapping of armyworm (Spodoptera frugiperda) in maize at a regional scale. Pest Manag. Sci. 2015, 73, 335–348. [Google Scholar]
  41. Buchaillot, M.L.; Cairns, J.; Hamadziripi, E.; Wilson, K.; Hughes, D.; Chelal, J.; McCloskey, P.; Kehs, A.; Clinton, N.; Araus, J.L.; et al. Regional monitoring of fall armyworm (FAW) using early warning systems. Remote Sens. 2022, 14, 5003. [Google Scholar] [CrossRef]
  42. Information on Korean Alien Species. Spodoptera exigua; National Institute of Ecology: Seochon, Korea, 2022; Available online: https://kias.nie.re.kr/ (accessed on 20 April 2023).
  43. Park, Y.-L.; Krell, R.K.; Carroll, M. Theory, technology, and practice of site-specific insect pest management. J. Asia-Pac. Entomol. 2007, 10, 89–101. [Google Scholar] [CrossRef]
  44. Park, Y.-L.; Krell, R.K. Generation of prescription maps for curative and preventative site-specific management of bean leaf beetles (Coleoptera: Chrysomelidae). J. Asia-Pac. Entomol. 2005, 8, 375–380. [Google Scholar] [CrossRef]
  45. Oliveira, R.A.; Näsi, R.; Niemeläinen, O.; Nyholm, L.; Alhonoja, K.; Kaivosoja, J.; Jauhiainen, L.; Viljanen, N.; Nezami, S.; Markelin, L.; et al. Machine learning estimators for the quantity and quality of grass swards used for silage production using drone-based imaging spectrometry and photogrammetry. Remote Sens. Environ. 2020, 246, 111830. [Google Scholar] [CrossRef]
  46. Kamilaris, A.; Prenafeta-Boldú, F.X. A review of the use of convolutional neural networks in agriculture. J. Agric. Sci. 2018, 156, 312–322. [Google Scholar] [CrossRef] [Green Version]
  47. Dias, P.A.; Tabb, A.; Medeiros, H. Apple flower detection using deep convolutional networks. Comp. Ind. 2018, 99, 17–28. [Google Scholar] [CrossRef] [Green Version]
Figure 1. Damage of soybean caused by S. exigua from the ground (A,B) and aerial (C,D) views. (A), a larva feeding on soybean; (B), a typical sign of soybean damage caused by S. exigua; (C,D), aerial views of soybean damage at 5 and 50 m above the ground, respectively.
Figure 1. Damage of soybean caused by S. exigua from the ground (A,B) and aerial (C,D) views. (A), a larva feeding on soybean; (B), a typical sign of soybean damage caused by S. exigua; (C,D), aerial views of soybean damage at 5 and 50 m above the ground, respectively.
Insects 14 00555 g001
Figure 2. UAS operation and image analysis protocol used in this study. Each step above was automated by autopilot for UAS flight and macro or batch processing for image analysis.
Figure 2. UAS operation and image analysis protocol used in this study. Each step above was automated by autopilot for UAS flight and macro or batch processing for image analysis.
Insects 14 00555 g002
Figure 3. Two different UAS flight plans for rapid assessment of soybean damage caused by S. exigua. (A), UAS flight path for an autopilot operation at 50 m above the ground; (B), a low-altitude flight path to confirm soybean defoliation by S. exigua at 5–10 m above the ground.
Figure 3. Two different UAS flight plans for rapid assessment of soybean damage caused by S. exigua. (A), UAS flight path for an autopilot operation at 50 m above the ground; (B), a low-altitude flight path to confirm soybean defoliation by S. exigua at 5–10 m above the ground.
Insects 14 00555 g003
Figure 4. Example aerial survey images at two different altitudes: a low-altitude flight at 5–10 m above the ground for validation aerial survey to confirm the presence of defoliation (A) and a high-altitude flight for aerial mapping of soybean defoliation by S. exigua at 50 m above the ground (B).
Figure 4. Example aerial survey images at two different altitudes: a low-altitude flight at 5–10 m above the ground for validation aerial survey to confirm the presence of defoliation (A) and a high-altitude flight for aerial mapping of soybean defoliation by S. exigua at 50 m above the ground (B).
Insects 14 00555 g004
Figure 5. A composite image (outlined in red) showing 31 soybean blocks and damage by S. exigua. The composite image is overlaid onto a base map in the geographic information system (GIS). See Table S1 for detailed information on the field size and amount of soybean damage.
Figure 5. A composite image (outlined in red) showing 31 soybean blocks and damage by S. exigua. The composite image is overlaid onto a base map in the geographic information system (GIS). See Table S1 for detailed information on the field size and amount of soybean damage.
Insects 14 00555 g005
Figure 6. Example aerial photos of soybean fields taken by UAS. Typical soybean fields (AC) with varying defoliation levels: (A), low defoliation; (B), high defoliation; (C), near 100% defoliation. Aerial views of healthy (D) and damaged (F) soybean fields and the two soybean blocks side by side with various levels of soybean defoliation by S. exigua (E).
Figure 6. Example aerial photos of soybean fields taken by UAS. Typical soybean fields (AC) with varying defoliation levels: (A), low defoliation; (B), high defoliation; (C), near 100% defoliation. Aerial views of healthy (D) and damaged (F) soybean fields and the two soybean blocks side by side with various levels of soybean defoliation by S. exigua (E).
Insects 14 00555 g006
Figure 7. The finalized damage map by image analyses, which extracted foliage of soybean plants (red) in the study area.
Figure 7. The finalized damage map by image analyses, which extracted foliage of soybean plants (red) in the study area.
Insects 14 00555 g007
Figure 8. Comparison of estimated survey hour (A) and estimated cost (B) between aerial survey and ground survey. * indicates a technology fee including rental for the UAS and an image analysis tool.
Figure 8. Comparison of estimated survey hour (A) and estimated cost (B) between aerial survey and ground survey. * indicates a technology fee including rental for the UAS and an image analysis tool.
Insects 14 00555 g008
Figure 9. NDVI values (darker green indicates healthier vegetation, and darker red indicates unhealthier vegetation) before (A), during (B), and after (C) the S. exigua outbreak. Areas with red color in (D) indicate soybean fields with S. exigua damage calculated from the difference in NDVI values between (A,B). Areas with green color (E) indicate soybean fields with recovery, regrown, or replanted after the S. exigua outbreak, which was calculated from the difference in NDVI values between (B,C). Areas with blue color (F) show the overlapping areas of (D,E), indicating the areas were damaged by S. exigua but also recovered or replated after the damage. An area with a black outline is the site for this study.
Figure 9. NDVI values (darker green indicates healthier vegetation, and darker red indicates unhealthier vegetation) before (A), during (B), and after (C) the S. exigua outbreak. Areas with red color in (D) indicate soybean fields with S. exigua damage calculated from the difference in NDVI values between (A,B). Areas with green color (E) indicate soybean fields with recovery, regrown, or replanted after the S. exigua outbreak, which was calculated from the difference in NDVI values between (B,C). Areas with blue color (F) show the overlapping areas of (D,E), indicating the areas were damaged by S. exigua but also recovered or replated after the damage. An area with a black outline is the site for this study.
Insects 14 00555 g009
Table 1. Geostatistical description of soybean damage by S. exigua. Note that soybean blocks not shown in this table had >95% damage, so their pattern was not analyzed.
Table 1. Geostatistical description of soybean damage by S. exigua. Note that soybean blocks not shown in this table had >95% damage, so their pattern was not analyzed.
BlockModelNuggetSillDD (%)Range (m)r2RSS
Block 1Exp.14,400212,60093.2890.927.53 × 108
Block 7Exp.101,000416,40075.72810.988.00 × 108
Block 8Sph.91,000494,10098.16060.983.22 × 109
Block 9Sph.100149,70099.91290.863.20 × 108
Block 13Sph.15,93038,32058.42720.951.08 × 107
Block 18Exp.48,100344,30086.02570.981.49 × 109
Block 19Gauss.99,200223,00055.51110.945.73 × 108
Block 20Exp.41,000401,00089.81660.934.59 × 109
Block 25Sph.15,200118,90087.22570.942.57 × 108
Block 26Exp.194,000560,40065.48720.971.09 × 109
Block 28Sph.100150,60099.91630.847.82 × 108
Block 29Sph.100149,70099.91290.863.20 × 108
Block 30Exp.810053,55084.91370.982.39 × 107
Block 31Sph.991025,11060.57520.871.72 × 107
DD, degree of spatial dependence; RSS, residual sums of squares; Exp., exponential model; Sph., spherical model; Gauss., Gaussian model.
Table 2. SADIE parameters for the distribution soybean damage caused by S. exigua in each soybean block surveyed by UAS. Note that soybean blocks not shown in this table had >95% damage, so their pattern was not analyzed.
Table 2. SADIE parameters for the distribution soybean damage caused by S. exigua in each soybean block surveyed by UAS. Note that soybean blocks not shown in this table had >95% damage, so their pattern was not analyzed.
BlockIaPa v j v i P v j P v i
Block 11.6040.0095−1.4951.4450.01180.0213
Block 71.8460.0015−1.8831.7880.00210.0036
Block 83.2720.0003−2.1902.1900.00000.0003
Block 92.1320.0015−1.5821.5480.00740.0131
Block 132.0300.0008−1.9111.8150.00100.0038
Block 185.7420.0003−1.8782.1350.00030.0000
Block 191.4820.0633−1.4441.3360.06790.1074
Block 201.5180.0051−1.6591.5630.00150.0038
Block 251.5340.0279−1.5331.3770.03180.0667
Block 262.7360.0003−1.9101.8830.00030.0003
Block 281.4170.0287−1.4131.2570.02870.0918
Block 292.1320.0015−1.5821.5480.00740.0131
Block 302.7740.0003−2.8162.5100.00000.0000
Block 311.7020.0074−1.6101.5030.01440.0290
Ia and Pa, index of aggregation and associated p value; vj and vi, and Pvj and Pvi, indices of clustering and associated p values.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Park, Y.-L.; Naharki, K.; Karimzadeh, R.; Seo, B.Y.; Lee, G.-S. Rapid Assessment of Insect Pest Outbreak Using Drones: A Case Study with Spodoptera exigua (Hübner) (Lepidoptera: Noctuidae) in Soybean Fields. Insects 2023, 14, 555. https://doi.org/10.3390/insects14060555

AMA Style

Park Y-L, Naharki K, Karimzadeh R, Seo BY, Lee G-S. Rapid Assessment of Insect Pest Outbreak Using Drones: A Case Study with Spodoptera exigua (Hübner) (Lepidoptera: Noctuidae) in Soybean Fields. Insects. 2023; 14(6):555. https://doi.org/10.3390/insects14060555

Chicago/Turabian Style

Park, Yong-Lak, Kushal Naharki, Roghaiyeh Karimzadeh, Bo Yoon Seo, and Gwan-Seok Lee. 2023. "Rapid Assessment of Insect Pest Outbreak Using Drones: A Case Study with Spodoptera exigua (Hübner) (Lepidoptera: Noctuidae) in Soybean Fields" Insects 14, no. 6: 555. https://doi.org/10.3390/insects14060555

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop