1. Introduction
Quantifying wildlife abundance is a critical aspect of ecological and conservation research, providing essential data for managing and protecting species. Accurate estimates of wildlife populations are crucial for understanding species distribution, assessing the impact of environmental changes, and informing conservation strategies. Among the various methods used to estimate wildlife abundance, line transect sampling is widely recognized for its efficiency and reliability. Line transect sampling is a widely used method in ecology for estimating the abundance or density of organisms within a particular habitat. The method involves laying a transect line (usually a long, straight line) through the habitat and recording the distance and location of all individuals or groups of individuals observed along the line. The data collected during line transect sampling can be used to estimate the density or abundance of the population being studied, using a variety of statistical methods. These methods typically involve estimating the probability of detection for each individual or group observed along the transect line and using this probability to correct the observed counts for incomplete detection. Line transect sampling can be conducted using a variety of techniques, including walking the transect line, driving along the transect line, or flying over the habitat in an aircraft or drone. The method can be used to study a wide range of organisms, including plants, insects, birds, mammals, and fish.
One of the distinct advantages of line transect sampling is its capability to efficiently collect data on a large number of individuals or groups within a relatively short period of time. This method provides valuable insights into population densities and distributions across different habitats. However, like any sampling technique, line transect sampling comes with its inherent limitations that must be considered. A primary limitation is the requirement for a straight transect line that accurately represents the habitat being studied. Deviations from this ideal can introduce biases in population estimates, particularly if habitats are irregular or fragmented. Moreover, the method’s reliance on linearity may overlook spatial complexities and variations in habitat structure that influence species distributions. Observer bias is another critical consideration. If the same observer collects all the data, their individual perception and experience can inadvertently influence data collection, potentially leading to biased estimates. Standardizing observer protocols and employing multiple observers can help mitigate this issue. Furthermore, line transect sampling may underestimate the abundance or density of certain organisms. Some species may be cryptic or have behaviors that make them difficult to detect along the transect line, especially if they occur in low densities or blend into their surroundings. This detection limitation can skew population estimates, particularly for rare or elusive species. Despite these challenges, line transect sampling remains a valuable tool in ecological research, providing quantitative data that contribute to conservation efforts, habitat management, and species monitoring. Addressing these limitations through rigorous study design, standardized protocols, and complementary sampling methods can enhance the reliability and applicability of line transect data in ecological studies.
The detection function, also known as the conditional function
, is used by Buckland et al. [
1] to express the model mathematically. The definition of the detection function is:
This truncation could happen either before or after the survey. The distance beyond which no additional observations are recorded is known as the greatest perpendicular distance, or
w. To evaluate the effectiveness of a detection model, several assumptions are made regarding the function
(see Buckland et al. [
1] and Miller and Thomas [
2]). It is expected that as the distance
x increases, the probability
of detecting an object randomly decreases monotonically, reflecting the higher likelihood of missing animals further away from the transect line. Mathematically, if
, then
. Additionally, items that are exactly on the transect path will always be visible (i.e.,
), which denotes perfect detection along the transect path. The detection probability should also approach one as the distance approaches zero, indicating high detection probability in close proximity (see Eberhardt [
3]). Since the slope at
is zero (i.e.,
), indicating a flat slope at zero distance, the tangent to the function is a horizontal line at zero distance. Consequently, these requirements are satisfied by the shape of
. The shoulder conditions are the set of presumptions that any detection model needs to take into account. The following traits are used by Buckland [
4] and Buckland et al. [
5] to characterize line transect sampling:
N items are dispersed at random throughout area A of the study zone, with an abundance of , the average population density.
The likelihood that an object is spotted on the line is 1, hence .
In the first place of sighting, objects are found.
There is only one count per item.
Perpendicular distances are measured carefully and accurately.
The distribution of objects occurs independently of the line.
There will be a few, possibly a lot, of missed objects.
Researchers Burnham and Anderson [
6] and Seber [
7] found that the following expression was correlated with the population density
D of objects in a given area:
In this case,
is a parameter that guarantees that
, for
, is a probability density function (PDF) for the perpendicular distance data.
L is the length of the transect line.
is the anticipated value of detections.
, which is dependent on an unknown parameter and belongs to a known family of functional forms, is estimated using the perpendicular distances (see Burnham and Anderson [
6]). The following describes the relationship between this PDF and
:
where the normalization factor
ensures that
is scaled appropriately. If and only if
, then
satisfies the shoulder criteria, according to Eberhardt [
8]. It is easy to confirm this equivalency. Thus,
has the same form as
but is scaled to have an area under the curve equal to one. It follows a monotonically declining trend. In relation to the suggested model for
, this scaling is essential for a precise estimation of
(see Crain et al. [
9]).
Other techniques that can be used to determine the population density
D include:
In this research, we assume
for simplicity’s sake. Nevertheless,
w can be substituted with a big enough value for an accurate approximation in real-world scenarios. The total number of objects detected is indicated by the parameter
n, and the appropriate estimate for
is
. In the literature, numerous parametric and nonparametric techniques for estimating
have been suggested. See the works of Burnham et al. [
10], Quinn and Gallucci [
11], and Pollock [
12] for additional developments in this methodology. The parametric approach is the main emphasis of this study. Specifically, the maximum likelihood technique is used to estimate parameters within the parametric detection model, which allows the estimator for
and the population abundance parameter
D to be estimated.
The article’s remaining sections are arranged as follows: In
Section 2, we suggest a negative Rayleigh detection model (NRDM) that meets the shoulder criteria and construct the probability density function (PDF) that goes along with it. This section also introduces some notable features of the suggested model.
Section 3 explores the moments of the NRDM and discusses related measures associated with it.
Section 4 focuses on the maximum likelihood estimations of the detection model parameters and presents the corresponding confidence intervals. In
Section 5, we employ Mathematica 10 for simulation purposes. We provide plots and graphs to visually examine the performance of the estimated parameters. The simulation allows us to calculate the population abundance along with its confidence interval. A practical dataset of perpendicular distance is also utilized to illustrate the utility of the NRDM. Under the suggested model, we obtain pertinent measurements for this collection of data. Finally, in
Section 6, based on the study’s findings, we give our concluding thoughts and insights.
2. The Proposed Detection Model
In fact, a less accurate estimator may result from the one-parameter model ’s potential inability to accurately represent the true form of the detection curve. We provide an enhanced detection function for the negative Rayleigh detection model (NRDM) in order to overcome this constraint.
The following detection function defines the NRDM:
The detection function can be utilized to characterize the detection probability along the transect path if it satisfies the shoulder criteria. An object is detected with certainty and probability one, or , when it is immediately on the transect route.
The detection function’s first derivative with regard to
x looks like this:
A more versatile model for determining the detection probability is the one-parameter NRDM,
u. The upcoming sections of the paper delve into the specifics of this model, encompassing its attributes and techniques for estimation. This implies that for
,
. Given that
monotonically decreases for each
, these shoulder requirements are further supported by a variety of forms (see
Figure 1).
The matching PDF of the NRDM is derived as a normalized
.
The PDF plots for various parametric values are displayed in
Figure 2. Equation (
6)’s first derivative with respect to
x is
Indeed, expression (
7) shows a direct proportionality between
and
. As a result,
and
exhibit similar characteristics, including
for
and a monotone decreasing shape. This implies that the proposed model can serve as a reliable estimator for
, satisfying the “Shape Criterion” as described by Burnham et al. [
10]. The reliability of the model is demonstrated in
Figure 1 and
Figure 2, which highlight the wide range of shapes that the detection function
can take, as it adjusts
based on the parameter
u. Furthermore, it is worth noting that all plots exhibit a progressive decay to zero as
, a desirable characteristic for the considered detection model.
Now, let us talk about some NRDM statistical data. The parameter
of Equation (
6) is first obtained by
and the population parameter
D is
The estimation of the population abundance D relies on estimating the parameter . This parameter is crucial in the estimation process. In order to estimate and subsequently estimate D, a parametric form (shape) for the detection function (or PDF) is required. Therefore, the approach used to estimate D is referred to as a parametric method. This method involves specifying a specific parametric form for the detection function, which allows for the estimation of the parameter and ultimately the estimation of D. The parametric method provides a structured framework for estimating population abundance based on the chosen parametric model for the detection function.
3. Some Statistical Properties
The proposed detection function ’s rth moments of the PDF are derived in this part, and they aid in the estimation of the parameter , allowing us to calculate the population abundance.
The following formula can be used to determine the
rth moment of a random variable
X given the probability density function (PDF) given in (
6):
After some algebra, we have
The mean and variance can be obtained by using
as
and
For some parameter values, the central tendency, variability, asymmetry, and peakedness can be described, where
denotes the
rth moment of the random variable
X,
represents the mean of
X, and
stands for the variance of
X. According to Ghitany et al. [
13] and Wackerly et al. [
14], skewness and kurtosis provide insights into the asymmetry and the shape of the distribution, respectively.
Table 1 presents the numerical values for the mean, variance, skewness, and kurtosis of
X for different parameter values of
u whose R code is given in
Appendix A. The graphical representation of these statistics is shown in
Figure 3. It is observed that as the parameter values increase, these statistical measures typically decrease. However, for any fixed value of
u, the skewness and kurtosis remain constant.
5. Simulation Study and Results
A simulation study evaluating the effectiveness of several estimators of the detection function is presented in this section. The suggested model estimators were the maximum likelihood (ML) estimator and the negative Rayleigh detection model (NRDM) estimator , which was computed using the method of moments’ estimator . We also took into account already-existing estimators, namely, the Weighted Exponential Model (WEM) estimator , the Half-Normal Model (HNM) estimator , and the Negative Exponential Model (NEM) estimator .
The simulation relied on two alternative target detection functions: the Exponential Power (EP) detection function and the Beta Exponential (BE) detection function. The simulation was run with various sample sizes and truncated points for each detection function. To replicate the perpendicular distances, two distinct target detection functions and observations
of
with sample sizes of
, 100, 200, 300, and 500 were taken into consideration. These detection functions were chosen based on the requirement that they reflect the range of shapes that could occur in the specified field (see Eidous [
19]). Perpendicular distances were simulated using the target models listed below:
The Exponential Power (EP) detection function, introduced by Pollock [
12], provides a flexible model for detection probability and is defined as:
In this function, denotes the gamma function, which is integral in various statistical distributions.
Eberhardt’s Beta Exponential (BE) detection function [
8] models detection probability with the formula:
This function is particularly useful in scenarios where the detection probability decreases linearly.
To mimic the data, every model was clipped at certain distances
w. The truncated points of the detection functions for BE and EP were
and
, respectively. For both models, a number of (arbitrary) values of
were chosen. We evaluate sample sizes of
, 100, 200, 300, and 500 for each model, while the samples for the randomly generated perpendicular distances were 1500 times larger. The results of the simulation research, along with the relative bias (RB) and relative mean square error (RME), are shown in
Table 2 and
Table 3.
The main conclusions drawn from the simulation study were as follows:
All estimators exhibited a decreasing relative bias (RB) and relative mean error (RME) as the sample size increased, indicating that the estimators become more reliable with larger sample sizes.
The ML estimator consistently outperformed the NRDM estimator in terms of RB and RME. This suggests that is a superior estimator compared to .
For all considered target detection functions, the suggested estimators demonstrated a reduced RB and RME in comparison to the NEM, HNM, and WEM estimators. This shows that the suggested estimators are better than the alternatives.
We plotted the RMEs for
,
,
, and
, respectively, in
Figure 4,
Figure 5,
Figure 6 and
Figure 7. These numbers allowed us to conclude that our suggested estimations were reasonable and performed exceptionally well for all target detection functions that were taken into consideration.
6. Model Accordance and Practical Data Utilization
Since model selection establishes the framework for data analysis and prediction, it is in fact an essential step in the modeling process. A thorough analysis of the data’s properties, including its distribution, variability, and possible sources of error, is necessary to select the best model. It also entails evaluating the trade-off between the accuracy of the model’s representation of the data and its complexity. The Akaike Information Criterion (AIC) and the Bayesian Information Criterion (BIC) are two examples of information criteria that can be used for model selection, in addition to likelihood-based and Bayesian approaches. By offering a numerical representation of the model’s goodness of fit and a penalty for model complexity, these techniques enable us to compare models with various numbers of parameters.
But there is no one-size-fits-all approach to model selection; rather, the best approach relies on the specifics of the research topic and the properties of the data. Expert judgment and topic expertise may occasionally be required to reach a final decision. After a model has been chosen, it is crucial to validate it to make sure it can accurately fit new data and is not overfitting the original data. Model validation entails evaluating the model’s goodness of fit using a variety of statistical metrics, including mean squared error, R-squared, and cross-validation, and testing the model on a different dataset that was not used for model fitting.
6.1. Comparative Models
The suggested detection model’s performance was compared to a number of other detection models using various goodness-of-fit evaluations. The models that were used for comparison are displayed in
Table 4.
6.2. Data on the Perpendicular Length in Meters
Dataset: Measurements in meters for parallel wooden stake distances.
The dataset contains 67 stakes collected along a single path of length 1000 m, from a population of 150 stakes. The true density is given as 0.00375 stake/meter. Based on earlier research by Karunamuni and Quinn [
25] and Zhang [
26], the value of
was given as 0.11029. The sagebrush meadow’s zone size was calculated to be 40,000
using the equation
, where
stakes.
Table 5 presents the data, which have been reviewed and reported by Burnham et al. [
10], Barabesi [
27] and Strindberg et al. [
28]. In contrast to the other values, the perpendicular distance of 31.31 in
Table 5 considerably departs from the transect path. Therefore, it is suggested to remove this extreme value or truncation point from the dataset to improve the accuracy of the estimation, as mentioned in Zhang [
26]. Based on this information, the decision was made to eliminate the mentioned extreme value or truncation point from the dataset to ensure the accuracy and reliability of the estimation process.
Table 6 provides the basic statistics of the wooden stakes’ dataset and presents the ratio of skewness to kurtosis for the wooden stakes’ dataset. This ratio provides insights into the distribution’s shape and helps assess how well the data align with theoretical expectations of the negative Rayleigh detection model (NRDM). The box plot and TTT plot in
Figure 8 offer more detail.
The study used maximum likelihood estimators, or MLEs, to ascertain the probability density function (PDF) of each detection model and assess how well it matched the collected data. Commonly used criteria used to compare the fit performance of models considered for the study of real datasets include the
p value of the Kolmogorov–Smirnov test, the Bayesian information criterion (BIC-1), the Hannan-Quinn information criterion (HQIC-1), the Akaike information criterion (AIC-1), and the corrected AIC (CAIC-1). They are shown in
Table 7 and
Table 8, and their Mathematica code is given in
Appendix B. Our model had the lowest information criteria values and the highest
p value for the Kolmogorov–Smirnov test, as can be seen. Therefore, it can be concluded that the proposed model is effective in this scenario.
Given the information provided, an analysis was conducted to estimate the population abundance (
) and the maximum likelihood estimate of the true density,
, using the maximum likelihood estimators (MLEs) from many studied models. The sample standard deviation is shown as
, while the theoretical standard deviation of the models is indicated by
. The
and
results are shown in
Table 9. The table shows that the NRDM (nonlinear Rayleigh detection model) model had the smallest absolute difference between the sample standard deviation (
) and the theoretical standard deviation (
) when using the p-value, K-S statistic, W*, A*, and MLEs. Furthermore, among the detection models that were tested, the estimates of
and
produced by the NRDM model were fairly close to the true values, which were reported to be 0.1127 and 0.0037, respectively, for the data under consideration. These findings were consistent with the true values of
and
D discovered in previous studies. Finally, with only minor deviations between the sample standard deviation and theoretical standard deviation, the NRDM model yielded the best fit to the true values of
and
D for the examined data.
Table 9 provides a summary of these results.
Further proof of the NRDM’s competitiveness can be found in
Figure 9, which plots the estimated survival functions and PDFs of the compared models for the considered dataset based on each model’s survival function and PDF after their parameters were replaced with the corresponding MLEs from
Table 7.